Collaborative Score Distillation for Consistent Visual Editing
Authors: Subin Kim
Kyungmin Lee
June Suk Choi
Jongheon Jeong
Kihyuk Sohn
Jinwoo Shin
Venue: NeurIPS 2023 poster (2023)
Keywords: Score Distillation Sampling
Diffusion model
Editing
TL;DR: Consistent visual editing across various and complex visual modalities

Abstract: Generative priors of large-scale text-to-image diffusion models enable a wide range of new generation and editing applications on diverse visual modalities. However, when adapting these priors to complex visual modalities, often represented as multiple images (e.g., video or 3D scene), achieving consistency across a set of images is challenging. In this paper, we address this challenge with a novel method, Collaborative Score Distillation (CSD). CSD is based on the Stein Variational Gradient Descent (SVGD). Specifically, we propose to consider multiple samples as “particles” in the SVGD update and combine their score functions to distill generative priors over a set of images synchronously. Thus, CSD facilitates the seamless integration of information across 2D images, leading to a consistent visual synthesis across multiple samples. We show the effectiveness of CSD in a variety of editing tasks, encompassing the visual editing of panorama images, videos, and 3D scenes. Our results underline the competency of CSD as a versatile method for enhancing inter-sample consistency, thereby broadening the applicability of text-to-image diffusion models.

Collaborative Score Distillation for Consistent Visual Editing
Subin Kim*
Kyungmin Lee Kibyuk Sohn
June Suk Choi
Jongheon Jeong'
Jinwoo Shin
KAIST
Google Research
"(subin-kim, kyungmnlee
Abstract
Generative priors of large-scale text-to-image diffusion models enable wide range of new generation and editing applications on diverse visual modalities. However, when adapting these priors to complex visual modalities, often represented as multiple images (e-g., video or scene), achieving consistency across a set images is challenging. In this paper, we address this challenge with novel method, Collaborative Score Distillation (CSD). CSD based on the Stein Variational Gradient Descent (SVGD). Specifically, we propose to consider multiple samples as "particles" in SVGD update and combine their score functions to distill generative priors over set of images synchronously. Thus, CSD facilitates the seamless integration of information across 2D images, leading to consistent visual synthesis across multiple samples. We show the effectiveness of CSD in variety editing tasks, encompassing the visual editing of panorama images, videos, and 3D scenes. Our results underline the competency of CSD versatile method for enhancing inter-sample consistency, thereby broadening the applicability text-to-image diffusion models.'
Introduction
Text-to-image diffusion models [1, 2, 4] have been scaled up by using billions of image-text pairs [5, 6] and efficient architectures [7, 8, 9, 4], showing impressive capability synthesizing high-quality, realistic, and diverse images with the text given branched into various applications, such as image-to-image translation [10, 11, 12, 13, 14, 15, 16], controllable generation [17], or personalization [18, 19]. One of the latest applications in this regard to translate the capability into other complex modalities, viz., beyond 2D images [20, 21] without modifying diffusion models using modality-speeiiff training data. This paper focuses the problem of adapting the knowledge of pre-trained text-to-image diffusion models to more complex high- dimensional visual manipulation tasks beyond 2D images without modifying diffusion models using modality-specific training data.
input. Furthermore, they have
We start from an intuition that many complex visual data, e.g., videos and 3D scenes, are represented as set of images constrained by modality-spectie requiring temporal consistency, and a 3D scene is set of multi-view frames with view consistency. Unfortunately, image diffusion models not have a built-in capability to ensure consistency between set of images for synthesis or editing because their generative sampling process does not take into account the consistency when using the image diffusion model as is. As such, when applying image diffusion models to manipulate these complex data without consistency in consideration,
consistency. For example, video a set of frames
"Equal contribution. Visualizations are available at the website htttpp::/suuin--eeee cv. www.gg.tt.bio..o io/CSD.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

Perometric
R g(a,)
Source,
Beyond sigle image
eg. panorame image, scone
k(x x
&e
"Tam inte
Difluiion Li Neti
Diffuson U--Wet
Diffusion Net
style poininng
Diffusion (et
Difluton Uitk 2
Diffusion U-Net
+il
(N)
Source
AE,
N.
Collaborative Distillation (CSD)
Backpropagate
Tozooz Bels (0,)
((())) AE; M(N
Edited
Figure I: Method overview. CSD-Edit enables various Squal-to-visual translations with two novel components. First, new score distillation scheme using Stein variational gradient descent, which considers inter-sample relationships (Section 3.1) to synthesize a of images while preserving modality-speciin consistency constraints. Second, our method edits images with minimal information given from text instruction by subtracting image-conditional noise estimate instead of random noise during score distillation (Section 3.2). By doing CSD-Edit is used for text-guided manipulation of various visual domains, e.g., panorama images, videos, and 3D scenes (Section 3.3).
results in a highly incoherent output, as in Figure (Patch-wise Crop), where one can easily identify where images are being stitched. Such behaviors are also reported in video editing, thus, recent works [22, 23, 24, 25] propose to handle video-specific temporal consistency when using the image diffusion model.
Here, we take attention to an alternative approach, Score Distillation Sampling (SDS) [26], which enables the optimization of arbitrary differentiable operators by leveraging the rich generative prior of text-to-image diffusion models. SDS poses generative sampling as an optimization problem by distilling the learned diffusion density scores. While Poole et al. [26] has shown the effectiveness of SDS in generating 3D objects from the text by resorting on Neural Radience Fields [27] priors which inherently suppose coherent geometry in 3D space through density modeling, it has not been studied for consistent visual manipulation of other modalities, where modality-specinnn consistency constraints should be considered when manipulating.
In this paper, we propose Collaborative Score Distillation (CSD), a simple yet effective method that extends the singular of the text-to-image diffusion model for consistent visual manipulation. The crux of our method is two-fold: first, we establish a generalization of SDS by using Stein variational gradient descent (SVGD), where multiple samples share their knowledge distilled from diffusion models to accomplish inter-sample consistency. Second, we present CSD-Edit, an effective method for consistent visual editing by leveraging CSD with Instruct-Pix2Pix [14], a recently proposed instruction-uuide image diffusion model (See Figure 1).
We demonstrate the versatility of our method in various editing applications such as panorama image editing, video editing, and reconstructed 3D scene editing. In editing a panorama image, we show that CSD-Edit obtains spatially consistent image editing by optimizing multiple patches of an image. Also, compared to other methods, our approach achieves a better trade-off between source-target image consistency and instruction fidelity. In video editing experiments, CSD-Edit obtains temporal consistency by taking multiple frames into optimization, resulting temporal frame-consistent video editing. Furthermore, apply CSD-Edit to 3D scene editing and generation, by encouraging consistent manipulation and synthesis among multiple views.
Preliminaries
2.1 Diffusion models
Generative modeling with diffusion models consists of a forward process y that gradually adds Gaussian noise to the input xo (x), and reverse process p which gradually denoises from

---

Spetial Consistency
Instruction Fidelity
Tarat--eall Copp
Instrac-PxIPix Patch-wise Crop
X
Instrctt PoZPix Overlapping
Iecln Cherleptngg Patrhan, -55
Inttus-Roo7lc Develageing Petches m11
Patches
(Ours)
ESP-Ri (Dues), 25
CDest (Oum),
Tort into Van-bogh style painting"
Figure 2: Panorama image editing. (Top right) Instruct-Pix2P19 [14] cropped patches results in inconsistent image edits. (Second row) Instruct-Pix2Pix on overlapping patches edits to a consistent image, but less fidelity to the instruction, even with high guidance scale wiy. (Third row) CSD-Edit provides consistent image editing with better instruction-fideliyy by setting a proper guidance scale.
the Gaussian noise Xy (0, I). Formally, the forward process gxxx//xxe. Xn) at timestep given by ((x, (xo) (xx;;; Xo I)), where and o? = 1 of are pre-defined constants designed for effective modeling [8, 28, 29]. Given enough timesteps, reverse process also becomes a Gaussian and the transitions are given by posterior with optimal MSE denoiser [30], i.e., (x (xx)
(xx - (x,; t), II where (x,;;)) t) is a learned optimal MSE denoiser. Ho et al. [7] proposed train U-Net [31] autoencoder (x,x;;; t) by minimizing following objective:
Loise (a:te(0:33) x) 1=-220001,,e~N((...
<l|2], = 0;X0 0,6
(1)
where w(t) is weighting function for each timestep t. Text-to-image diffusion models [1, 2, 4, 3] trained by Eq. (I) with (x(xx;2, () that estimates the noise conditioned on the text prompt To effectively guide the text-conditional generation, Ho et al. [32] proposed classifier-fre guidance (CFG), where they jointly train the unconditional and conditional model and interpolate the unconditional and conditional model during the inference, i.e., the noise estimate is given by
ex (x y, t) es (x;: t) + wy (cs((x)))
ex(xx:
where 2 is guidance scale that controls the sample fidelity. Specifically, increasing wy enhances By sample fidelity at the expense of sample diversity. Throughout the paper, we refer p (xx; y. conditional distribution of text y.
Instruction-based image editing by instruct-Pix2P1 Recently, many works have demonstrated
the capability of diffusion models in editing stylizing images [10, 13, 11, 12, 14]. Among them, Brooks et al. [14] proposed Instruct-Pix Pix2Pix, where they finetuned Stable Diffusion [4] model with the source image. text instruction, (edited) target image (edited by Prompt-to-Promee
[12]) triplets to
enable instruction-based editing of an image. Then during the inference,
Siiruutt-PxxPPii starts from
source image and conducts diffusion sampling by the diffusion model that takes instruction y. In specific, given the source image & and instruction the noise estimate at time is given by
(x:: X. y. (x(:)) + wa (Es x, x, Es (x(: t))
+ ww (E X, t) (x(;;; X, x,t). t)).
where Wy 2 is the CFG parameter for text instruction in Eq. (2) and W, is additional CFG parameter that controls the fidelity to the source image x.
Score distillation sampling
Poole et al. [26] proposed Score Distillation Sampling (SDS), alternative sample generation method by distilling the rich knowledge text-to-image diffusion models. SDS allows optimization of any differentiable image generator, e.g., Neural Radiance Fields [27] the image space itself.

---

Source
FateZero
Pbx2Video
Gen-1
CSD-Edit (Ouns)
"Make it as a painting of Claude Monet"
Figure 3: Video editing. Qualitative results on the lucia video in DAVIS 2017 [33]. CSD shows frame-wise consistent editing providing coherent content across video frames e.g., consistent color and background without changes in person. Compared to Gen-l [21], video editing method trained on a large video dataset, CSD-Edit shows high-quality video editing results reflecting given prompts.
Formally, let x = g(0) be an image rendered by a differentiable generator with parameter 6, then SDS minimizes density distillation loss [34] which is variational inference via minimizing KL divergence between the posterior of x g(0) and the text-conditional density yg:
min Coierina (0:x g(0) = Er,e la//r DEa (q xxxx/x==x 9(0)) P (x t))]
(4)
where X, 0,X o,E with g(0) and (O. They derive SDS by differentiating Eq. with respect to generator parameter 0, but omitting the U-Net Jachoian term due to its poor performance and computationally inefficient. The SDS gradient update is given follows:
0x
V L (0:x = g(0) = E.,e (E
c) ae
(5)
In its implementation, we randomly sample timestep from uniformly distributed interval U fain fmax] The range of timesteps tzin and faax chosen sample from not too small or large noise levels, and the guidance scales are chosen to be larger than those used for image generation.
2.3 Stein variational gradient deseent
The original motivation of Stein variational gradient descent (SVGD) [35] is to solve a variational inference problem, where the goal is to approximate a target distribution from a simpler distribution by minimizing KL divergence. Formally, suppose is target distribution with known score function Vx log p(x) that we aim to approximate, and q(x) is known source distribution. Liu and Wang [35] showed that the steepest descent of KL divergence between q and is given as follows:
Byyx)) [f(x) Vx logp(x) + fxx)),
where f FR R is any smooth vector function that satisfies -i fxx) 0. Remark that Eq. (6) becomes zero if we replace q(x) with p(x) in the expectation term, which is known as

---

Instruct- Searuct-NRF2Neeee
CSD-Edit (Ours)
Original NeRF
(a) "What he were anime character?
(b) "Make him smile"
Figure 4: 3D NeRF scene editing. Visualizing novel-views of edited Fangzhou NeRF scene [37]. CSD-Edit leads to high-quality editing of 3D scenes and better preserves semantics of source scenes, e.g., obtains sharp facial details (left) and makes him smile without giving beard (right).
Stein's identity [36]. Here, the choice of the critic is erucial its convergence and computational tractability. To that end, Liu and Wang [35] proposed to constrain f in the Reproducing Kernel Hilbert Space (RKHS) which yields closed-form solution. Specifically, given positive definite kernel RDD
k: R
+ Rt, Stein variational gradient descent provides the greedy directions as follows:
x - x nAx,
Ax = Eg(xe) [k(x, x' Vx log p(x) + xxx,(x x'),
with small step size 1j > O. The SVGD update in Eq. (7) consists of two terms that play different roles: the first term moves the particles towards the high-density region of target density p(x), where the direction is smoothed by kernels other particles. The second term acts as repulsive force that prevents the mode collapse of particles. One choose different kernel functions, while resort to standard Radial Basis Function (RBF) kernel x() exp(- - h Xx x' x'112) with bandwidth >> 0.
Method
In this section, we introduce Collaborative Score Distillation (CSD) for consistent synthesis and editing of multiple samples. We first derive a collaborative score distillation method using Stein variational gradient descent (Section 3.1) and propose an effective image editing method using CSD, i... CSD-Edit, that leads to coherent editing of multiple images with instruction (Section 3.2). Lastly, we present various applications of CSD-Edit editing panorama images, videos, and 3D scenes (Section 3.3).
3.1 Collaborative score distillation
Suppose of parameters (0: J that generates images (())
=0(0)) Similar SDS, our goal is
to update each 0, by distilling the smoothed densities from the diffusion model by minimizing KI.
divergence in Eq. (4), On the contrary, CSD solves Eq. (4) using SVGD demonstrated in Section 2.3
so that each 0, can be updated in sync with updates of other parameters in the set [ At each update, CSD samples U(taia tnax) and NN I), and update each 0, follows:
Vo, Co.0
w() N
Σ 1=1
N
((x)))
1 x (E s..t) - e) +
((x
x(()) x()))
Ox
(8)
00;
for each i i 1,2, ....................... N. We refer to Appendix A for full derivation. Note CSD is equivalent to SDS Eq. (5) when N showing that CSD is generalization of SDS to multiple samples. As the pairwise kernel values are multiplied by the noise prediction term, each parameter update on 0, is affected by other parameters, i.e., the scores are mixed with importance weights according to the affinity among samples. The more similar samples tend to exchange more score updates, while different samples tend interchange the score information less. The gradient of the kernels acts as repulsive force that prevents the mode collapse of samples. Moreover, we note that Eq. (8) does not
5

---

make any assumption on the relation between 8, 's or their order besides them being a set of images to be synthesized coherently with each other. As such, CSD is also applicable to arbitrary image generators, as well as text-to-3D synthesis in DreamFusion [26], which we compare in Section 4.4.
3.2 Instruction-guided editing by collaborative score distillation
In this section, we introduce an instruction-guided visual editing method using Collaborative Score Distillation (CSD-Edit). Given source images x g(O,) with parameters 0,, we optimize new target parameters {0. with x( = g(0,) such that 1) each x(6) follows the instruction prompt, 2) preserves the semantics of source images as much as possible, and 3) the obtained images are consistent with each other. To this end, we update each parameter 9,, initialized with 6, using CSD with noise estimate E " of Pixiue [14]. However, this approach often results in blurred outputs, leading the loss of details of the source image (see Figure 6). This is because the score distillation term subtracts random noise e, which perturbs the undesirable details of source images.
We handle this issue by adjusting the noise prediction term that enhances the consistency between source and target images. Subtracting random noise e in Eq. (5) when computing the gradient is a crucial factor, which helps optimization by reducing the variance of gradient. Therefore, amend the optimization by changing the random noise into a better baseline function. Since our goal is to edit an image with only minimal information given text instructions, set the baseline by the image-conditional noise estimate c% of the Instruct-Pix2Pix model without giving text instructions on the source image. To be specific, our CSD-Edit is given as follows:
Vo, Cc50 Edit (0.)
wt(t
N
k(x)
10)) Xi
AE
+
V
(n 6) xx²)))
(x(1) 00,
N
AE
(1)
8 y. 1)
x t).
In Section 4.4, we validate our findings the effect of baseline noise image editing performance. We notice that CSD-Edit presents an alternative way to utilize Instruct-Pix2PP in image-editing without any finetuning of diffusion models, by posing an optimization problem.
3.3 CSD-Edit for various complex visual domains
Panorama image editing. Diffusion models are usually trained on a fixed resolution (c.g., 512x512 for Stable Diffusion [4]), thus when editing panorama image (i.c., image with a large aspect ratio), the editing quality significantly degrades. Otherwise, one can crop an image into smaller patches and apply image editing on each patch. However this results in spatially inconsistent images (see Figure 2, Patch-wise Crop, Appendix D). To that end, we propose to apply CSD-Edit on patches to obtain spatially consistent editing of image, while preserving the semanties of the source image. Following [38], we sample patches of size 512x512 that overlap using small stride and apply CSD-Edit on the latent space of Stable Diffusion [4]. Since we allow overlapping, some pixels might be updated more frequently. Thus, we normalize the gradient of each pixel by counting the appearance. Remark that one can give different instructions on the different regions of an image while maintaining consistency (See Appendix C for details).
Video editing. Editing a video with an instruction should satisfy the following: 1) temporal consistency between frames such that the degree of changes compared to the source video should be consistent across frames, 2) ensuring that desired edits in each edited frame are in line with the given prompts while preserving the original structure of source video, and 3) maintaining the sample quality in each frame after editing. To meet these requirements, we randomly sample a batch of frames and update them with CSD-Edit to achieve temporal consistency between frames.
3D scene editing. We consider editing a 3D scene reconstructed by a Neural Radiance Field (NeRF) [27], which represents volumetric 3D scenes using 2D images. To edit reconstructed 3D NeRF scenes, it is straightforward to update the training views with edited views and finetune the NeRF with edited views. Here, the multi-view consistency between edited views should be considered since inconsistencies between edits across multiple viewpoints lead to blurry and unde- sirable artifacts, hindcring the optimization of NeRF. To mitigate this, Haque et al. [39] proposed

---

Inferior
Superior
Instruction Fidelity
Table 1: Video editing, Quantitative comparison of CSD-Edit with baselines on video editing. Bold indicates the best results.
0.9
GS=3
GS=5
Image Similarity
GS=7.5
0.8
GS-10
CLIP Directional
CLIP Image
LPIPS
Similariy
Consistency
FateZero [22]
0.312
0.948
Pix2Video [25]
0.229
0.948
0.282
CSD-Edit (Ours)
0.319
0.957
0.235
0.6
CSD-Edit (Ours)
Table 2: 3D scene editing, Quantitative compari- son of CSD-Edit with baselines on 3D scene edit- ing. Bold indicates the best results.
Instruct Pix2Pix Overlapping Instruct-Rxx2Px Downscaling
0.5
0.05
0,10
0.15 0.20 0.25
CLIP Directional Similarity t
Image
LPIPS 4
CLIP Directional Similarity
Consistency
Figure 5: Panorama image editing. Compar- ison of CSD-Edit with baselines at different guidance scales wy E [3.0,55 5.0, 7.5, 10.0).
IN2N CSD-Edit (Ours)
a.177 0.215 uLas2
0.993axxx:
0.053
0.994.
0.045 4232
Instruct-NeRF2NeRRF which performs editing on a subset of training views and updates them se- quentially at training iteration with intervals. However, image-wise editing results in inconsistencies between views, thus they rely on the ability of NeRF in achieving multi-view consistency. Contrary to Instruct-NeRe NeRF2N we update the dataset with multiple consistent views through CSD-Edit, which serves as better training resources for NeRF, leading to less artifacts and better preservation of source 3D scene.
4 Experiments
4.1 Text-guided panorama image editing
For the panorama image-to-image translation task, we compare CSD-Edit with different versions of Instruct-Pix2Pix: one is which using naive downsizing to 512 512 and performing Instruct- Pix2Pix, and another is updating Instruct-Pix2Pix on the patches cropped with overlapping as in MultiDiffusion [38] Instruct-Pix211
+ Overlapping). For comparison, we collect set of panorama images (i.e., which aspect ratio is higher than 3), and edit each image to various artistic styles and different guidance scales wy. For evaluation,
pre-trained CLIP [40] to measure two different metrics: 1) consistency between source and target images by computing similarity between two image embeddings, and 2) CLIP directional similarity [41] which measures how the change in text agrees with the change in the images. The experimental details are in Appendix B.I.
In Figure 5, we plot the CLIP scores of different image editing methods with different guidance scales. We notice that CSD-Edit provides the best trade-off between the consistency between source and target images and fidelity to the instructions. Figure provides qualitative comparison between panorama image editing methods. Note that applying cestruct-PixxPiie to patches cropped with overlapping
EusstructPiixPPi, + Overlapping) is able to edit images while preserving spatial consistency, as evidenced by high CLIP image similarity, however, the edited images show inferior fidelity the text instruction directional similarity. We conjecture that this happens because the scores are diluted by other scores, i.e. one patch may respond much more or much less the instruction compared others. Additional qualitative results are in Appendix D.
when using large guidance scale, resulting in much lower CL.IP
4.2 Text-guided video editing
For the video editing experiments, we primarily compare CSD-Edit with existing zero-shot video editing schemes that employ text-to-image diffusion models such as FateZero [22] and Pix2Video [25]. To emphasize the effectiveness of CSD-Edit over learning baaend schemes, we also compare it with Gen-l [21], a state-of-the-an video editing method trained on a large video dataset. For quantitative evaluation, we report CLIP image-text directional similarity as in Section 4.1 to measure the alignment between changes in text and images. We also measure CLIP image consistency and L.IIS [42] between consecutive frames to evaluate temporal consistency, In addition to the objective metrics
7

---

seite. ewa
tee ewz.co
ewz.ch jr.
rzz chiffoo fobs
ch Jobs
cit Jobs
sette
awa
te awz.co
ewz wwz.ch//aa
wz cnitots
cc//job
ch/loos
LEVd
Lins
& Land
Source
SDS (Random noise subtraction)
Sa tte. ewz.c
ewz.ch wz
rz ch fobs
ch jobs
hhionh
selte. ewa ite,
ewz.cl
ewz.ch/jo Jo
wz.ch uz.ch/joobs
Lch Lch/joba
ch jobs
Do
DEG
ii
nee
of
Without SVGD
CSD-Edit
"Give him a cap"
Figure 6: Ablation study. Given source video (top left), CSD-Edit without SVGD results in inconsistent frames (bottom left), and replacing the subtraction of image-conditioned noise in CSD- Edit to the subtraction of random noise results in loss of details and original structures (top right). CSD-Edit obtains and preserves consistency between edits without loss of semantics (bottom right).
for evaluating consistency and instruction fidelity, we also conduct a user study, given the subjective nature of editing tasks. We measure the user rankings for temporal consistency between edited frames, frime-wise instruction fidelity, and the editing quality, respectively. We use video sequences from the popular DAVIS [33] dataset at resolution of 1920 1080. Please refer to Appendix B.2 and Appendix C.3 for detailed description of the baseline methods and experimental setup.
Table and Table 6 in Appendix C.3 summarize a quantitative comparison between CSD-Edit and the baselines. We note that CSD-Edit consistently outperforms the existing zero-shot video editing approaches in terms of both temporal consistency and fidelity to given text prompts. Furthermore, Figure 3 qualitatively demonstrates the superiority of CSD-Edit over the baselines. Specifically, CSD-Edit maintains consistent style across all frames for both the woman and the background elements (e.g., bench, trees), ensuring a consistent degree of editing throughout the video. On the other hand, FateZero and Pix2Video result in noticeably inconsistent edits from frame to the next. Impressively, CSD-Edit not only demonstrates temporally consistent edits compared to Gen-1, but also excels preserving the original semantics of the source video, even without training video dataset and without requiring any architectural modifications to the diffusion model. Additional qualitative results, including video stylization and object-aware editing tasks, in Appendix D.
large
4.3 Text-guided 3D scene editing
For the text-guided 3D scene editing experiments, we mainly compare our approach with Instuct- NeRF2NeRF (IN2N) [39]. For a fair comparison, we exactly follow the experimental setup which they used, and faithfully find the hyperparameters to reproduce their results. For evaluation, we render images at the novel views (i.e., views not seen during training), and report CLIP image similarity and LPIPS between consecutive frames rendered videos to measure multi-view consistency, as well as CLIP image-text similarity to measure fidelity to the instruction. In addition, we conduct user studies to evaluate the multi-view consistency, caarucctio-fidellii and the editing quality, respectively. Detailed explanations for each dataset sequence and training details can be found in Appendix B.3.
Figure 4, Table 2, and Table in Appendix C.3 summarize the comparison between CSD-Edit and IN2N. We notice that CSD-Edit enables a wide-range control of 3D NeRF scenes, such as delicate attribute manipulation (e.g., facial expression alterations) and acene-stylization (c.g., conversion to the animation style). Especially, we notice two advantages of CSD- compared to IN2N. First, CSD-Edit presents high-quality details the edited 3D scene by providing multi-view consistent training views during NeRF optimization. In Figure 4, one can observe that CSD-Edit captures sharp details of the anime character, while IN2N results in a blurry face. Second, CSD-Edit better at preserving the semantics of source 3D scenes, e.g., backgrounds colors. For instance in Figure 4, we notice that CSD-Edit allows subtle changes facial expressions without changing the color of the background adding a beard to the face.

---

Table 3:
Ablation on video editing. Ablation study of CSD-Edit by removing SVGD or replac- ing the subtraction of the image-conditioned noise
Superior
CUP Similarity
0.8
into the random noise (i.e. SDS).
0.7
CLIP Directional
CLIP Image Consistency
LPIPS
Similarity 1
Spatial Consistency
CSD-Edit
0.320
0.957
0.236
Without SVGD
0.278
0.909
0.372
0.5
SDS [26]
0.270
0.884
0.316
CSD Edit
0.4
Without SVGD
SDS
Inferior
Table Ablation on 3D scene editing. Ablation
study of CSD-Fdit by removing SVGD or replac- ing the subtraction of the image-conditiones noise into the random noise (ie., SDS).
0.10
0.15
0.20
0.25
0.30
CLIP Directional Similarity
Figure 7: Ablation on panorama image edit- ing. Ablation study of CSD-Edit by removing SVGD or replacing the subtraction of image- conditioned noise into the random noise (SDS).
CLIP Directional
CLIP Image
LPIPS
Similarity t
Consistency f
+
CSD-Edit Without SVGD
0.239
0.995
0.043
0.186
0.993
0.053
SDS [26]
0,186
0.994
0.053
4.4 Ablation study
To demonstrate the effectiveness of our method, CSD-Edit, we conduct ablation studies in each visual domain. To verify the role of Stein Variational Gradient Descent (SVGD) in ensuring consistency during the editing, we remove the kernel mixing component in Eq. (9). In addition, to verify the role of the subtracting image- conditioned noise estimate (E in Eq. (9)) in obtaining high-quality visual edits, we subtract the Gaussian noise that used to forward samples as similar to SDS [26], instead of the image-conditioned noise estimate. Again, we follow the experimental and evaluation setups in Section 4.1, Section 4.2, and Section 4.3 for the editing of each visual domain, respectively.
Figure 7, Table 3, and Table 4 show the quantitative results of the ablation studies on panorama image editing, video editing, and 3D scene editing, respectively. Note that the absence of SVGD in CSD-Edit radically alters the image, underscoring its critical role in consistency regularization. This is evidenced by lower CLIP image similarity scores, which measure the consistency between the source and edited images, and lower LPIPS scores, which evaluate the consistency between edits. In addition, the lack of mixing scores between samples leads abrupt and undesirable changes in the source, as shown in Figure 8. Further, replacing image-conditioned noise subtraction with random noise subtraction results in a loss of the original structure of the source image, which significantly degrades CLIP image similarity and CLIP directional similarity.
In Figure 6, provide qualitative validation of the effectiveness of method on video editing. CSD-Edit consistently edits a source video by adding a red cap to a man's 's head when given the instruction "Give him cap." However, with SVGD, the edits between frames inconsistent, e.g. both blue and red caps appear on the edited frames. In addition, if we set the baseline noise as the random noise injected into the source and target images, each frame becomes blurred and loses the original structures, such as blurred legs and backgrounds. For more qualitative results of our ablation studies, please refer to Figure 13 in Appendix D.
5 Related work
Following the remarkable success of text-to-image diffusion models [4, 20, 2, 43], numerous works have attempted to exploit rich knowledge them for various visual editing tasks including images [10, 44, 13, 45, 14, 12, 15], videos [46, 25]. 3D scenes [39], etc. However, extending existing image editing approaches to more complex visual modalities often faces a new challenge; consistency between edits, e.g., spatial consistency in high-resolution images, temporal consistency in videos, and multi-view consistency in 3D scenes. While prior works primarily focus on designing task-specific methods [24, 22, 25] or model fine-tuning for complex modalities [46], we present codaliit--gnosseee novel method for editing, effectively capturing consistency between samples.
The most related to our work is DreamFusion [26], which introduced Score Distillation Sampling (SDS) for the creation of 3D assets, leveraging the power text-to-image diffusion models. Despite the flexible merit of SDS to enable the optimization of arbitrary differentiable operators, most

---

"Turn sheeps into tigers"
(a) Source
(b) CSD-Edit without SVGD
(c) CSD-Edit
Figure 8: Ablation study for the effect of SVGD. Our kernel mixing scores in CSD-Edit act as regularizer that prevents abrupt changes in images, ensuring better consistency: the tiger is generated in the unwanted region of a source image when SVGD is not applied in the update.
works mainly focus on applying SDS to enhance the synthesis quality of 3D scenes by introducing 3DD-seciffic frameworks [47, 48, 49, 50, 51]. Although there exists some work to apply SDS for visual domains other than 3D assets, they have limited their scope to image editing [52]. or image generation [53]. Here, we argue that the current main challenge preventing the wider application of SDS, especially in higher-dimensional visual manipulations beyond single 2D images at fixed resolutions, is the lack of control over inter-sample consistency. To the best of our knowledge, our work is the first to identify this challenge and to lay the novel foundations for principled adaptation of text-to to-ieaee diffusion models to more diverse and high- nigh-dimensional visual manipulations.
6 Conclusion
In this paper, we propose Collaborative Score Distillation (CSD) for consistent visual synthesis and manipulation. CSD is built upon Stein variational gradient descent, where multiple samples share their knowledge distilled from text-to-image diffusion models during the update. Furthermore, we propose CSD-Edit that gives us consistent editing of images by distilling minimal, yet sufficient information from instruction-guiic method in text- ext-g translation of diverse visual contents, such as in high-resolution images, videos, and real 3D scenes, outperforming previous methods both quantitatively and qualitatively.
diffusion models. We demonstrate the effectiveness of our
Limitations and future works. Since results are often imperfect due to the inherent inability of diffusion models to understand language. Furthermore, our method relies generative priors derived from large text-to-image diffusion models, which may inadvertently contain biases due to the auto-filtering process applied to the vast training dataset. However, we believe that employing Consistent Score Distillation (CSD) can assist us in identifying and understanding such undesirable biases. By leveraging the inter-sample relationships and aiming for consistent generation and manipulation of visual content, our method provides a valuable avenue for comprehending the interaction between samples and prompts. Further exploration of this aspect represents an intriguing future direction.
pre-trained text-to-image diffusion models, obtained
In addition, although our primary interest is the editing (not the generation) of panoramic images, videos, 3D scenes, believe that CSD has the potential to be used in their generation. As presented in Section C.2 in Appendix C, we show how CSD can improve generation performance over SDS [26] text-to-3D generation experiments. In particular, we verify the effect of CSD in improving the geometry and quality of text-to-3D generation. In this sense, exploring this aspect of synthesis with CSD could be an interesting research topic and we leave it for future work.
Societal impact. Our research introduces a comprchensive image editing framework that encom- passes various modalities, including high-resolution images, videos, and 3D scenes. While it is important to acknowledge that our framework might be potentially misused to create fake content, this concern is inherent to image editing techniques whole. We expect future research on the detection of generated visual content.
10

---

Acknowledgement
This work was supported by Institute of Information communications Technology Planning & Evaluation (IIPP) grant funded by the Korea governmen((MSIT) NN.2009-0-000777 Artificial Intelligence Graduate School Program(KAIST); Hub; No. No.2022-0-001 --00044,, Development and Study of AI Technologies to Inexpensively Conform to Evolving Policy on Ethics). This work is in partly supported by Google Research grant and Google Cloud Research Credits program. We thank Jason Baldridge, Yi-Hsuan Tsai, Sihyun Yu, and Sangwoo Mo for reviewing our manuscript, and anonymous reviewers for their helpful comments and suggestions.
Artificial Intelligence Innovation
References
[1] C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gon- tijo Lopes. B. Karagol Ayan, T. Salimans, et al. Photorcalistic text-to-image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 35: 36479-36494, 2022.
[2] A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen. Hierarchical text-conditional image generation with CLIP latents. arXiv preprint arXiv:2204.066/55, 2022.
[3] A. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. McGrew, I. Sutskever, and M. Chen. GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741 2021.
[4] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684-10695, 2022.
[5] C. Schuhmann, R. Vencu, R. Beaumont, R. Kaczmarczyk, Mullis, A. Katta, T. Coombes, J. Jitsev, and A. Komatsuzaki. LAION-400m: Open dataset of CLIP-filterd 400 million image-text pairs, arXiv preprint arXiv:2111.02114, 2021.
[6] C. Schuhmann, R. Beaumont, R. Vencu, C. Gordon, R. Wightman, M. Cherti, T. Coombes, A. Katta, C. Mullis, M. Wortsman, al. LAION-5b: An open large-scale dataset for training next generation image-text models. arXiv preprini 10.0022210008-42, 2022.
[7] J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6855 68511 2020.
[8] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint a3X4:2001.11345. 2020.
[9] J. Song, C. Meng, and S. Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020.
[10] C. Meng, Y. Song, J. Song, J. Wu, J.-Y, Zhu, and S. Ermon. SDEdit: Image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.011 01073, 2021.
[11] O. Bar- Tal, D. Ofri-Amar, R. Fridman, Y. Kasten, and T. Dekel. Tex12LIVE: Text-driven layered image and video editing. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XV, pages 707-723. Springer, 2022.
[12] A. Hertz, R. Mokady. J. Tenenbaum, K. Aberman, Y. Pritch, and D. Cohen-Or. Prompt-to- prompt image editing with cross attention control. arXiv preprint arXiv:2008. 01626, 2022.
[13] B. Kawar, S. Zada, O. Lang, O. Tov, H. Chang, T. Dekel, Mosseri, and M. Trani. Imagic: Text-based real image editing with diffusion models. arXiv preprint arXiv:2210009276 2022.
[14] T. Brooks, Holynski, and A. A. Efros. Instructpix2pix: Learning to follow image editing instructions. arXiv preprint arXh::2211.09800 2022.

---

[15] R. Mokady, A. Hertz, K. Aberman, Y. Pritch, and D. Cohen- Or. Null-text inversion for editing real images using guided diffusion models. arXiv preprint arXiv:2211.09740 2022.
[16] A. Voynov, K. Aberman, and D. Cohen-Or. Sketch-guided text-to-image diffusion models. arXiv preprini urXiv:2211.13752. 2022.
[17] L. Zhang and M. Agrawala. Adding conditional control to text-to-image diffusion models. arXiv preprint arXiv:2302.05543 2023.
[18] R. Gal, Y. Alaluf, Y. Atzmon, O. Patashnik, A. H. Bermano, G. Chechik, and Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208. 01618, 2022.
[19] N. Ruiz, Y. Li, V. Jampani, Y. Pritch, M. Rubinstein, and K. Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. arXiv preprint 2Xiv:2208.1224: 2022.
[20] J. Ho. W. Chan, C. Saharia, J. Whang, R. Gao, A. Gritsenko, D. Kingma, B. Poole, M. Norouzi, D. J. Fleet, et al. Imagen video: High definition video generation with diffusion models. arXiv preprint 2XXiv:2210.02303, 2022.
[21] Esser, J. Chiu, P. Atighehchian, J. Granskog, and A. Germanidis. Structure and content-guided video synthesis with diffusion models. arXiv preprint arXiv:2302.0301 2023.
[22] C. Qi, Cun, Y. Zhang, C. Lei, X. Wang, Shan, and Q. Chen. Fatezero: Fusing attentions for zero-shot text-based video editing. arXiv preprint arXiv:2303.09555,, 2023.
[23] L. Khachatryan, A. Movsisyan, V. Tadevosyan, R. Henschel, Z. Wang, S. Navasardyan, and H. Shi. Text2Video-Zero: Texl-to-image diffusion models are zero-shot video generators. arXiv preprint arXiv:2305.13333 2023.
[24] S. Liu, Y. Zhang, W. W. Lin, and J. Jia. Video-p2p: Video editing with cross-attention control. arXiv preprint arX1v:2303.04761, 2023.
[25] D. Ceylan, C.-H. Huang, and N. J. Mitra. Pix2Video: Video editing using image diffusion. arXiv preprint arXiv:2303. /2688, 2023.
[26] B. Poole, Jain, T. Barron, and B. Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209. 14988, 2022.
[27] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamorthi, and R. Ng. NeRF: Representing scenes as neural radiance fields for view synthesis. In European Conference on Computer Vision, 2020.
[28] D. Kingma, T. Salimans, B. Poole, and Ho. Variational diffusion models. Advances in neural information processing systems, 144:1696-211 2021.
[29] T. Karras, M. Aittala, T. Aila, and S. Laine. Elucidating the design space of iiffusion-based generative models. arXiv preprint arXiv:2206.00364, 2022.
[30] J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In F. Bach and D. Blei, editors, Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pages 2256-2265, Lille, France, 07-09 Jul 2015. PMIR. URL https: //proceedings. mlr. waa..ppess/vvie v37 http:/ooohdiickteeiiiiiie ckstein15....... html.
[31] O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pages 234-241. Springer, 2015.
[32] Ho and T. Salimans. Classifier-free diffusion guidance, arXiv preprint arXiv:2207.12595 2022.
12

---

[33] J. Pust-Tuset, F. Perazzi, S. Caelles, P. Arbeláez, A. Sorkine-Homung, and L. Van Gool. The 2017 DAVIS challenge on video object segmentation. 1XX4:1774.000755 2017.
[34] A. Oord, Y. Li, Babuschkin, K. Simonyan, O. Vinyals, K. Kavukcuoglu, G. Driessche, E. Lockhart, L. Cobo, F. Stimberg, et al. Parallel WaveNet: Fast high-fidelity speech synthesis. In International conference on machine learning, pages 3918-3926 PMLR, 2018.
[35] Q. Liu and D. Wang. Stein variational gradient descent: A general purpose bayesian inference algorithm. Advances in neural information processing systems, 29, 2016.
[36] Gorham and L.. Mackey. Measuring sample quality with kernels. International Conference on Machine Learning, pages 1292-1301. PMLR, 2017.
[37] C. Wang, R. Jiang, M. Chai, M. He, D. Chen, and J. Liao. NeRF-Art: Text-driven neural radiance fields stylization arXiv preprint 1exxiv2222.080077 2022.
[38] O. Bar-Tai L. Yariv, Y. Lipman. and T. Dekel. Multidiffusion: Fusing diffusion paths for controlled image generation. arXiv preprint wrXiv:2302.08113, 2023.
[39] A. Haque, M. Tancik, A. Efros, A. Holynski, and A. Kanazawa. Instruct-NeRrNNRR: Editing
3d scenes with instructions. In Proceedings of the IEEE/CVF International Conference Computer Vision, 2023.
[40] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, Mishkin, J. Clark, et Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763 PMLR, 2021.
[41] R. Gal, O. Patashnik, H. Maron, A. H. Bermano, G. Chechik, and D. Cohen-Or. Stylegan-nada: Clip-guided domain adaptation of image generators. ACM Transactions on Graphics (TOG), 41 (4):1-13, 2022.
[42] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The unreasonable effectiveness of deep features as perceptual metric. In CVPR, 2018.
[43] Y. Balaji, S. Nah, X. Huang, A. Vahdat, J. Song, K. Kreis, M. Aittala, T. Aila, S. Laine, B. Catanzaro, al, ediffi: Text-to-image diffusion models with an ensemble of expert denoisers. arXiv preprint arXiv:22/1.000 1.001324, 2022.
[44] G. Couairon, J. Verbeck, H. Schwenk, and M. Cord. Diffedit: Diffusion-based semantic image editing with mask guidance. arXiv preprint arXiv:2210. 11427, 2022.
[45] D. Valevski, M. Kalman, Y. Matias, and Y. Leviathan. Unitune: Text-driven image editing by fine tuning an image generation model on a single image. arXiv preprint arXiv:2210.094/ 2022.
[46] J. Z. Wu, Y. Ge, X. Wang, W. Lei, Y. Gu, W. Hsu, Y. Shan, X. Qie, and M. Z. Shou. Tune-a- video: One-shot tuning of image diffusion models for text-to-video generation. arXiv preprint 2121v:2212.11555, 2022.
[47] C.-H. Lin, J. Gao, L. Tang, T. Takikawa, X. Zeng, X. Huang. K. Kreis, S. Fidler, M.-Y. Liu, and T.-Y. Lin. Magic3d: High-resolution text-to-3d content creation. arXiv preprint arXiv:2211. 10440, 2022.
[48] C. Tsalicoglou, F. Manhardt, A. Tonioni, M. Niemeyer, and F. Tombari. Textmesh: Generation of realistic 3d meshes from text prompts. arXiv preprini arXiv:2304. 12439, 2023.
[49] L. Melas-Kyriazi, C. Rupprecht, I. Laina, and A. Vedaldi. Realfusion: 360 ('dde] reconstruction of any object from a single image. arXiv preprint arXiv:2302.1066. 2023.
[50] R. Chen, Y. Chen, N. Jiao, and K. Jia. Fantasia3d: Disentangling geometry and appearance for high-quality text-to-3d content creation. arXiv preprint 33373 2023.
[51] Tang, T. Wang, B. Zhang, T. Zhang, R. Yi, L. Ma, and D. Chen. Make-it-3d: High-fidelity 3d creation from single image with diffusion prior. arXiv preprint arXiv:2303. 2023,
13

---

[52] A. Hertz, K. Aberman, and D. Cohen-Or. Delta denoising score. arXiv:2304.07090 2023.
arXiv preprint
[53] K. Song, L. Han, B. Liu, D. Metaxas, and A. Elgammal. Diffusion guided domain adaptation of image generators. arXiv preprint arXiv:2212.04477, 2022.
[54] J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:/500.02433. 2015.
[55] R. Jordan, D. Kinderlehrer, and F. Otto. The variational formulation of the fokker-planck equation. SIAM journal on mathematical analysis, 29(1):1-17, 1998.
[56] G. Ilharco, M. Wortsman, R. Wightman, Gordon, N. Carlini, R. Taori, Dave, V. Shankar, H. Namkoong, J. Miller, H. Hajishirzi, A. Farhadi, and L. Schmidt. OpenCLIP, July 2021. URL
200..5211//00 5281/zenodo. 5143773.
[57] M. Tancik, E. Weber, E. Ng. R. Li, B. Yi, J. Kerr, T. Wang, A. Kristoffersen, Austin, K. Salahi, al. Nerfstudio: A modular framework for neural radiance field development. arXiv preprint arXiv:2332.044644. 2023.
[58] Y. Du, C. Durkan, R. Strudel, J. B. Tenenbaum, Dieleman, R. Fergus, J. Sohl-Dickstein, A. Doucet, and W. Grathwohl. Reduce, reuse, recycle: Compositional generation with energy- based diffusion models and meme. arXiv preprini ar0iv /1552, 2023.
[59] N. Liu, S. Li, Y. Du, A. Torralba, and J. B. Tenenbaum. Compositional visual generation with composable diffusion models. In Computer Vision-ECCY ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XVII, pages 423-439. Springer, 2022.
[60] J. Tang. https://github.. http:://gituub.com//ssawwww/ hwtp////wwwwwweee//bbeeeeettttee
Stable-dreamfusion:
Text-to-3d
with
stable-diffusion, 2022.
[61] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 770- 778, 2016.
[62] X. Xie, P. Zhou, H. Li, Z. Lin, and S. Yan. Adan: Adaptive nesterov momentum algorithm for faster optimizing deep models. arXiv preprint arXiv:2208.06677 2022.
[63] H. Wang, X. Du, J. Li, R. A. Yeh, and G. Shakhnarovich. Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation. CVPR, 2022.
14

---

Appendix
Website: https://subin-KRR http://uuiiikiicccc..... www.ii.gitiib................. 1o/CSD
A Technical details
In this section, we provide detailed explanations on the proposed methods, CSD and CSD-Edit.
CSD derivation. Consider a set of parameters {0: [0: which generates images x g(O,). For each timestep (((ain fnxx), we aim minimizing the following KL divergence
DxL (x(x) (x(1) (x
for each = 1, .. (i.e., gradient of log-density) by the noise predictor from diffusion model follows:
.N via SVGD using Eq. (7). To this end, we approximate the score function,
Ex(x iv,
log (x iv.
Then, the gradient of score function with respect to parameter 0; is given by
Ve, loggo (x, y. V x (x ;y, 1)
Ox
(6)
-
at
(x) iy,
Ox(
(10)
00;
00
for each i == L..... N. Finally, derive CSD, we plug Eq. (10) to Eq. (7) to attain Eq. (8). Also, we subtract the noise e, which helps reduce the variance of the gradient for better optimization. Following DreamFusion [26], we do not compute the Jacobian of U-Net. At high level, CSD takes the gradient update on each (9) using SVGD and updates 0; by simple chain rule without computing Jacobian. This formulation makes CSD a straightforward generalization to SDS for multiple samples and leads to an effective gradient for optimizing consistency among batches of samples.
CSD-Edit derivation. As mentioned above, we subtract the random noise reduce the variance of CSD gradient estimation. This is in similar manner to the variance reduction in policy gradient [54], where having a proper baseline function results in faster and more stable optimization. Using this analogy, our intuition is built upon that setting a better baseline function can ameliorate the optimization of CSD. However, as shown in Figures and Figure 13 in our manuscript, the default choice of SDS (random noise) results in highly blurred outputs, because the boise-denoising process of SDS blurs the image. Therefore, we propose to use image-conditions baseline function in image editing via CSD-Edit, that the diffusion noise only alters the part where the text instructs to change, by allowing CSD-Edit to optimize the latent driven only by the influence of the instruction prompts. This approach is supported by the principle of Wasserstein gradient flow [55], where the optimal gradient flow in variational inference is given by the difference between the target score function and the source score function.
noise estimation as a
Moreover, we notice that similar observations were proposed in Delta Denoising Score (DDS) [52], where they introduced an image-to-image translation method that is based on SDS, and the difference of the noise estimate from target prompt and that from source prompt are used. Our CSD can be combined with DDS by changing the noise difference term follows:
AE (x,; Vegt, t) c. (Xx: Vsrc: t),
where x and & are target and source images, V/ge and yare are target and source prompts. However, found that CSD-Edit with InstructPix2Pix more amenable in editing real images as it does not require a source prompt. Finally, we remark that CSD-Edit can be applied to various text-to-image diffusion models such as ControlNet [17], which we leave for future work.
15

---

B Implementation details
Setup. For the experiments with CSD-Edit, use the publicly available pre-trained model of Instruct-Pix2PP5 [14] by default. We perform CSD-Edit optimization on the output space of Stable Diffusion [4] autoencoder. We use SGD optimizer with step learning rate decay, without adding weight decay. We set min 0.2 and max = 0.5, where original SDS optimization for DreamFusion used fin 0.2 and laxx = 0.98. This is because we do not generally require a large scale of noise in editing. We use the guidance scale wy [3.0, 15.0] and image guidance scale w. [1.5, 5.0]. find that our approach less sensitive to the choice of image guidance scale, yet a smaller image guidance scale is more sensitive to editing. All experiments conducted on AMD EPYC 7V13 64-Core Processor and a single NVIDIA A100 80GB. Throughout the experiments, we use OpenCLIP [56] ViT-bigG- 14 model for evaluation.
B.1 Panorama image editing
To edit panorama image, we first encode into the Stable Diffusion latent space (i.e., downscale by 8), then use stride size of 16 to obtain multiple patches. Then we select a B batch of patches to perform CSD-Edit. Note that we perform CSD-Edit and then normalize by the number of appearances as mentioned in Section 3.3. Note that our approach performs well even without using small batch size, c.g., for an image of resolution 1120x512, there are 12 patches and we use =
For experiments, we collect 32 panorama images and conduct 5 artistic stylizations: "turn into Van Gogh style painting". "turn into Pablo Picasso style painting". "turn into Andy Warhol style painting". "turn into oriental style painting", and "turn into Salvador Dali style painting". We use learning rate of 2.0 and image guidance scale of 1 1.5, and vary the guidance scale from 3.0 to 10.0.
B.2 Video editing
We edit video sequences in DAVIS 2017 [33] by sampling 24 frames at the resolution of 1920 1080 from each sequence. Then, resize all frames into 512x 512 resolution and encode all frames each using Stable Diffusion. We use learning rate [0.25, 2] and optimize them for [200, 500] iterations.
B.3
3D scene editing
Following Sestruct-NRF2NNeke [39], first pretrain NeRF using the nerfacto model from NeRFS- tudio [57], training it for 30,000 steps. Next, we re-initialize the optimizer and finetune the pre-trained NeRF model with edited train views. In contrast to Instruct-NeRFNNeRR, which edits one train view with Instruct-Pix2Pin after every 10 steps of update, we edit a batch of train views (batch size of 16) with CSD-Edit after every 2000 steps of update. The batch randomly selected among the train views without replacement.
16

---

C More experimental results
C.1
Compositional editing
Recent works have shown the ability of text-to-image diffusion models in compositional generation of images handling multiple prompts [58, 59]. Here, we show that CSD-Edi can extend this ability to compositional editing, even at panorama-scake images which require a particular ability to maintain far-range consistency. Specifically, we demonstrate that one can edit a panorama image to follow different prompts on different regions while keeping the overall context uncorrupted.
Given multiple textual prompts {yx K k=1 the compositional noise estimate is given by
K aLE (xt: yk, t),
cx(xi; t)
where an are hyperparameters that regularize the effect of each prompt. When applying compositional generation to the panorama image editing, the challenge lies in obtaining image that smooth and natural within the region where the different prompts are applied. To that end, for each patch of an image, we set ag to be the area of the overlapping region between the patch and region where prompt yt is applied. Also, normalize to assure Σx ag 1. In Figure 10, illustrate some examples on compositional editing of a panorama image. For instance, given an image, one can change into different weathers, different seasons, different painting styles without leaving artifacts that hinder the spatial consistency of an image.
C.2 Text-to-3D generation with CSD
We explore the effectiveness of CSD in text-to-3D generation tasks following DreamFusion [26]. We train a coordinate MLP-based NeRF architecture from scratch using lext-to-image diffusion models. Since the pixel-space diffusion model that DreamFusion used [26] is not publicly available, we used an open-source implementation of pixel-space text-to-image diffusion model. Given a set of text prompts, we run both DreamFusion and DreamFusion with CSD with a fixed seed. Our experiments in this section are based on Stable-DreamFuss DreamFusion [60], public re-implementation of DreamFusion, given that currently the official implementation of DreamFusion is not available on public.
Setup. We use vanilla MLP based NeRF architecture [27] with 5 ResNet [61] blocks. Other regularizers such as shading, camera and light sampling are set as default in [60]. We use view- dependent prompting given the sampled azimuth angle and interpolate by the text embeddings. We use Adan [62] optimizer with learning rate warmup over 2000 steps from 10-8 to 2 x 10-3 followed by cosine decay down to 10-6. We use batch size of 4 and optimize for 10000 steps in total, where most of the case sufficiently converged at 7000 to 8000 steps. For the base text-to-image diffusion 0 since we found it way better than the default choice of Stable Diffuusion in a qualitative manner. While the original DreamFussonn [26] used guidance scale of 100 for their experiments, we find that guidance scale of 20 works well for DeepFloyd. We selected 30 prompts used in DreamFusion gallery* and compare their generation results via DreamFusion from the standard SDS and those from our proposed CSD. We use one A A100 (80GB) GPU for each experiment, and it takes ~55 hours to conduct one experiment. For CSD implementation, we use LPIPS [42] distance of RBF kemee.. Note that IPIIS gives more computational cost than the usual norm based RBF kernel. The LPIPS is computed between two rendered views of size 64. aed?
model, we adopt DeepFloyde -F---ooddd-----------
For the kernel bandwidth, we use h log B. where med is median of the pairwise LPIPS distance
between the views, B is the batch size.
For evaluation, we render the scene at the elevation 30 degree and capture at every 30 degree of azimuth angle. Then we compute the CLIP image-text similarity between the rendered views and input prompts. We measure similarities for both textured views (RGB) and textureless depth views (Depth). We also report Frechet Inception Distance (FID) between the RGB images and ImageNet validation dataset to evaluate the quality and diversity of rendered images compared to natural images.
hhatpee//github..com//eeppp floydd/
"betpe://arremfuuuiondd. .ww.adddgggiiiii github. wtw.lllrr....//l..............l. html
17

---

Table 5: Text-to-3D. Quantitative comparison between CSD and SDS under on text-to-3D generation via DreamFusion [26]
CLIP Similarity
CLIP Similarity
FID
Color t
Geo
SDS [26]
0.437
0.322
259.4
CSD (Ours)
0.447
0.345
247.1
Results. Table 5, we report the evaluation results of CSD on text-to-3D generation comparison to DreamFusion. Remark that CSD presents better CLIP image-text similarities in both RGB and Depth views. Also, CSD achieves lower FID score showing its better quality on generated samples. Since used the random seed in generating both CSD and DreamFusion, the shapes and colors similar. However, the results show that CSD obtains finer details its generations.
In Figure 14, qualitatively compare the baseline DreamFusion (SDS) and ours. We empirically observe three advantages using CSD over SDS, First, CSD provides better quality compared to SDS. SDS often suffers from Janus problem, where multiple faces appear in 3D object. We found that CSD often resolves Janus problem by showing consistent information during training. See the first row of Figure 14. Second, CSD give better distillation often gives us blurry artifact or undesirable features left in the 3D object. CSD can handle this problem and results in higher-quality generation, e.g., Figure 14 second row. Lastly, CSD can be used for improving diversity. One problem of DreamFusion. as acclaimed by the authors, that it lacks sample diversity. Thus, it often relies on changing random seeds, but it largely alters the output, On the other hand, we show that CSD can obtain alternative sample with only small details changed, e.g., Figure 14 third row. Even when SDS successful, CSD can be used in generating diverse sample.
quality. The inconsistent score
C.3
User study
Given the subjective nature of editing tasks, additionally conduct subjective user studies, where ask three questions in evaluating the editing quality of both CSD-Edit and haselines: the consistency of the edited results, frame-wise image instruction fidelity, and the editing quality. For each of the three studies, we asked 20 subjects to rank different methods. As shown in Table and Table 7, our method, CSD-Edit, consistently outperforms other baselines achieving the best user preferences across all three aspects.
Table User study on video editing.
Temporal Consistency
Instruction Fidelity
Editing Quality +
FateZero [22]
2.37
2.05
2.12
Vix2Video [25]
2.36
2.35
2.28
CSD-Edit (Ours)
1.27
1.6
1.6
Table 7: User study on 3D scene editing.
Temporal Consistency
Instruction Fidelity
Editing Quality
IN2N [14]
1.61
1.69
1.71
CSD-Edit (Ours)
1.39
1.31
1.29
18

---

C.4 Computation time comparison
Regarding the computational efficiency of our method, we measure the computation times and compare them with the baselines of each task. All these evaluations were conducted on a single NVIDIA A100 80GB and AMD EPYC 7V13 64-Core Processor.
For panorama image editing experiments, we compare our method with the baseline where applying patches cropped with overlapping (InstructPix2Pix Overlapping Patches). Note that the computation time both methods depends on the input image resolution, whereas we show that our method becomes more efficient the resolution goes higher. Table 8 shows how the computation time (total time in seconds) differs by the size of the input image. Here, the baseline method requires computing noise estimates of every patch each diffusion step, while our method only requires computing minibatch of patches per iteration.
Instruct-Pix2155
Table 8: Computation time comparison on panorama image editing.
Method
Resolution
Total time (sec.)
InstructPix2Pix + Overlapping Patches
1920x640
62
CSD-Edit (Ours)
1920x640
68
InstructPix2Pix + Overlapping Patches
3968x4352
487
CSD-Edit (Ours)
3968x4352
275
In video editing experiments, we measure the total computation time of our method and baseline methods. As shown in Table 9, our method, CSD-Edit consistently outperforms the baselines, taking the shortest time in editing (x3.3 times faster than FateZero [22], x1.3 times faster than Pix2Video [25]. Note that the computational efficiency of method, CSD-Edit, video editing can be easily improved by reducing the number of iterations for optimization while using a larger learning rate. Here, using a higher learning rate changes frames more radically, thus slightly degrading the frame consistency, however, the effect is meager as shown in CLIP Image consistency and LPIPS metries.
Table 9: Computation time comparison on video editing.
Total time (sec).
CLIP Directional
CLIP Image Consistency
LPIPS
Similarity f
FateZero [22]
192
0.312
0.948
0.264
Pix2Vidco [25]
77
0.229
0.948
0.28210001
CSD-Edit (Ours)
59
0.320
0.955
0.243
CSD-Edit (Ours)
423
0.319=0.00
0.957-t0.01
0.235 100001
In text-to-3D generation experiments, when directly comparing with DreamFusion [26] SJC [63]), there is slight increase in the computational cost due to the usage of LPIPS metric as distance for RBF kernel. For instance, takes hour generate 3D model with DreamFusion, while using CSD takes 84 minutes. We note that the increment is not substantial, however, ours yields better qualities with finer details, shown in Table 5 and Figure 14 the Appendix.
19

---

C.5 More analysis about the batch size
We denote N as the total number of images (e.g., total patches of panorama image or total frames of a video) and we update B minibatch of samples per iteration. Intuitively, using large B would encourage more consistency among samples, but more computationally expensive. Also, we observed that using large batch sizes dilutes the effect of editing. Thus, the batch size controls the trade-off between computation time, editing quality, and preserving consistency. further verify our choice of B, we provide additional ablation studies on the panorama image editing experiments. Given the same experimental setup as Section 4.1 with fixed guidance scale=7.5, we swept over == 4, 8, 12, and measured the CLIP source-target image similarity, CLIP directional image-text similarity, and computation time (iteration per second for total 200 iterations, measured on single A100 40GB GPU). Table 10 demonstrates the effect of batch size.
Table 10: Ablation study about the batch size.
Batch size
Total time (iter/ sec.)
CLIP Directional Similarity
CLIP Image Similarity
B=4
2.86
0.639
0.240
0.692
0.217
B=12
1.02
0,739
0,195
"Turn penguins into chickens" "
(a) minibatch B=3
(b) minibatch B=12
Figure 9: Ablation study: the effect of batch size. One can control the diversity of generated output, e.g., the same penguins are changed to more diverse chickens, by choosing appropriate batch size.
20

---

D
Additional qualitative results
Source
"Turn into sunny weather"
"Turn into rainy weather"
"Turn into snowy weather"
"Turn into spring"
"Turn into fall"
"Turn Gogh style painting"
"Turn into Paul Gauguin style painting"
Figure 10: Compositional image editing. CSD-Edit demonstrates the ability to edit consistently and coherently across patches in panorama images. This provides the unique capability to manipulate each patch according to different instructions while maintaining the overall structure of the source image. Remarkably, CSD-Edit ensures smooth transition between patches, even when different instructions are applied.
21

---

Source
"Turn sheeps into wolves"
"Turn sheeps into kangaroos"
"Turn sheeps into polar bears"
"Turn sheeps reindeers"
Source
"Turn penguins into chickens"
"Turn penguins into bears"
"Turn penguins into pandas"
"Turn penguins lions"
Figure 11: Object editing. CSD-Edit can edit many objects wide panorama image consistently accordance with the given instruction while preserving the overall structure of source images.
22

---

Source
Fatezero
Pix2Video
CSD-Edit (Ours)
"Turn a bear into tiger"
Source
FateZero
Pbx2Video
CSD-Edit (Ours)
"Give him yellow T-shirt"
Figure 12: Video editing. CSD-Edit demonstrates various editing from an object (e.g., tiger) to attributes (e.g., color) while providing consistent edits across frames and maintaining the overall structure of source video.
23

---

Source
"Turn into Van Gogh style painting"
"Turn into Poblo Picasso style painting"
"Turn into Andy Warhol style painting"
"Turn into oriental style painting"
"Turn into Salvador Dali style painting"
CSD-Edit
CSD-Edit without SVGD
CSD-Edit with Random Noise
Figure 13: Ablation study: SVGD and random noise. As illustrated, edits across different patches are not consistent without SVGD. Also, when using random noise as baseline noise, it loses the content and the detail of the source image.
24

---

"a fox holding videogome controller'
"a crocodile playing drum set
plush dragon toy"
"Chichen Itza, aerial view
"a beautiful dress made out of fruit, mannequin"
"a squirnel in samurai armor wielding katana"
SDS
CSD (Ours)
SDS
CSD (Ours)
Figure 14: Text-to-3D generation examples. (First row) CSD helps to capture coherent geometry compared to using SDS. (Second row) CSD allows learning finer details than SDS. (Third row) CSD can provide diverse and high-quality samples without changing random seeds.
25

---

Source
"Turn flowers into red roses"
Source
"Turn it into an old vintage photo"
Figure 15: Limitations. (First row) CSD-Edit often manipulates undesirable contents due to the inherent inability Setruct-Pix2Pii, model. (Second row) CSD-Edit often produces artifacts on the image due to the patch-wise wise update.
E
Limitations
As our method leverages pre-trained Instruct-Pix2Pix, it inherits the limitations of it such as unde- sirable changes the image due to the biases. Also, as described [14], Instruct-Pix2PP unable to change viewpoints, isolate specific object, or reorganize objects within the image.
often
When editing high-resolution image by dividing it into patches, it often remains artifact the edge of the patches, especially at the corner side of an image. This is due to that the patches at the corner less likely to be sampled during the optimization. See Figure 15 for examples.
When editing a video, the edited video often shows a flickering effect due to the inability of the Stable Diffusion autoencoder compress the video. We believe that using CSD-Edi Edit with video diffusion models trained on video datasets can possibly overcome this problem.
26


Reviews:
[{"review_id": "53ME9xl1pv", "reviewer": "Reviewer_v8pz", "summary": "This paper presents a novel method for achieving consistent visual synthesis using a diffusion model. Specifically, the authors extend Score Distillation Sampling to accommodate more complex visual modalities, represented as multiple images. They introduce a principled method to jointly optimize these multiple samples (referred to as particles), ensuring that each sample matches the distribution of an image (evaluated by a pre-trained diffusion model) while maintaining consistency with each other. This methodology is rooted in Stein Variational Gradient Descent. The authors successfully demonstrate a substantial performance improvement in visual quality and consistency across a diverse range of visual editing and synthesis tasks, including panorama editing, video editing, and 3D editing.\n\n\n", "questions": "This is not related to my evaluation. But could the author compare this paper with a concurrent paper ProlificDreamer [1]. Specifically, in Section 3.2, the authors mention the need to substitute the random noise with the score predicted by an unconditional model. I am interested in understanding whether this is related to the second noise prediction feature present in ProlificDreamer\n\n\n\n\n[1] Wang, Zhengyi, et al. \"ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation.\" arXiv preprint arXiv:2305.16213 (2023).", "limitations": "Yes. ", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "S1: The proposed method is well-founded and extends the original Score Distillation Sampling (SDS) concept to a novel setting that requires the simultaneous optimization of multiple images. Achieving consistency among different samples is non-trivial and often depends on domain-specific structure (e.g. nerf in DreamFusion, cross attention control in video synthesis). This paper introduces a more principled technique for consistent synthesis, marking a significant contribution to the field.\n\nS2: The overall presentation of the paper is commendable, with a clear structure and explanation that makes it easy to follow.\n\nS3: The method proposed is theoretically sound and delivers impressive results across a wide range of editing tasks. Besides its theoretical contribution, the method demonstrates strong practical applications, incorporating application-specific structure for panorama, 3D, and video. This demonstrates its potential as a solid baseline for future methods in the field.\n\n\n", "weaknesses": "While the current approach achieves a degree of consistency by jointly optimizing different images using Stein Variational Gradient Descent, it doesn't necessarily guarantee flawless consistency. This is evidenced by the presence of artifacts at patch boundaries and flickering between video frames. Therefore, consistent synthesis may still necessitate the incorporation of more general structures, such as volumetric rendering, in future visual synthesis pipelines (3D or Video Editing). ", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer v8pz, \n\nWe sincerely appreciate your thoughtful comments, efforts, and time to improve our manuscript. We respond to each of your questions and concerns one-by-one in what follows. Please let us know if you have any comments/concerns that we have not addressed up to your satisfaction.\n\n---\n\n**[W1] How to ensure flawless and complete consistency going further from proper consistency between a set of images?**\n\nAt a high level, CSD aims at preserving the consistency between the source images when updating with pre-trained text-to-image diffusion models. In principle, we cast this as a multi-particle variational inference problem and apply Stein \nVariational Gradient Descent algorithm. Through empirical validation, we show that CSD can generate images that follow the text instruction without breaking the consistency. However, we acknowledge that our method might not guarantee the `perfect flawless’ consistency. We also agree with your suggestion in that incorporating modality-specific generators, such as neural fields, or utilizing better priors to ensure consistency, e.g., optical flow for temporal consistency, could be definitely an interesting future work to explore when adapting text-to-image diffusion models to high-dimensional visual synthesis.  \n\n---\n\n**[Q1] Additional comparison to a concurrent work, ProlificDreamer [1] to understand the role of subtracted noise**\n\nThank you for bringing out the concurrent work, ProlificDreamer, which is very relevant to our paper! Both CSD and ProlificDreamer are based on similar methods, but the tasks of interest are quite different, leading to different choices of second noises for each: we are interested in manipulation of various modalities including images, videos, and 3D scenes, while ProlificDreamer only considers text-to-3D generation. In its basis, CSD and ProlificDreamer have identical objectives in distilling the generative prior of pre-trained text-to-image diffusion models using weighted KL divergence (i.e., Eq.(4) in our manuscript). Then the particle-based optimization using Wasserstein gradient flow [2] is given by the different between the score function of target distribution $p$ and source distribution $q$:\n$$d\\mathbf{x} = (\\nabla_{\\mathbf{x}} \\log p(\\mathbf{x}) - \\nabla_{\\mathbf{x}} \\log q_t(\\mathbf{x}))dt $$\nHere, the score of target distribution is given by a pre-trained diffusion model, while the score function of source distribution is not present as default. In our paper, we use the image-conditional noise estimate of InstructPix2Pix, which approximately estimates the score of source distribution. In ProlificDreamer, they resort to online fine-tuning of a diffusion model with respect to the current NeRF scenes. Notably, we additionally compute kernels and mix the scores of multiple samples, which enhances consistency. We believe that ProlificDreamer and CSD are complementary works that the idea of CSD can be used in ProlificDreamer to enhance 3D consistency in text-to-3D generation, which we leave as future work. \n\n---\n**Reference**\n\n\n[1] Wang et al., ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation, arXiv 2023\n\n[2] Richard Jordan, David Kinderlehrer, and Felix Otto. “The variational formulation of the Fokker–Planck equation”. In: SIAM journal on mathematical analysis 29.1 (1998). Publisher: SIAM, pp. 1–17.\n"}, {"review_id": "9o9wWF9dW2", "reviewer": "Reviewer_qUzm", "summary": "The paper presents a novel method, Collaborative Score Distillation (CSD), for diffusion models. The authors propose a new approach to score distillation that leverages the inter-sample relationships to generate more consistent and coherent images. The paper also introduces CSD-Edit, an extension of CSD, which enables the editing of images, videos, and 3D representations. The paper demonstrates the effectiveness of the proposed approach through extensive experiments, showing that the method outperforms existing methods in various tasks, including high-resolution image editing, video editing, and 3D scene editing.", "questions": "- Could the authors provide more details on the computational efficiency of their method, particularly in comparison with other score distillation methods such as SDS and SJC? It would be beneficial to understand the trade-offs between the quality of the results and the computational resources required.\n- How does the method handle errors or biases in the pre-trained Instruct-Pix2Pix model?\n- How is the number of samples (N) determined and selected in different experiments and applications (as per Eq. (8))? This appears to be a crucial hyperparameter in the proposed method, and a more detailed explanation of how it is chosen would be helpful.\n- In Section 4.4, how are view-dependent prompts and CSD unified and integrated for use in text-to-3D applications?\n- The multi-head (or Janus) problem is a well-known issue in text-to-3D applications. Can CSD be used to address this problem? If so, how effective is it, and are there any limitations or challenges in using CSD for this purpose?", "limitations": "The authors discuss some limitations of their approach in the supplementary material, but these could be more prominently addressed in the main paper. In particular, the issue of artifacts in high-resolution image editing and the flickering effect in video editing are significant limitations that should be discussed in more detail.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(+) The paper presents a novel approach to score distillation that takes into account the inter-sample relationships. This is a departure from existing methods, which typically focus on individual samples. The idea of using Stein Variational Gradient Descent (SVGD) to enforce consistency among samples shows promise in improving the quality of generated images. The ability to edit images, videos, and 3D could have wide-ranging applications in various fields.\n\n(+) The paper is well-written and the proposed methods are clearly explained.\n\n(+) The authors demonstrate the effectiveness of their approach through extensive examples (in the main paper, supplementary document, and webpage).", "weaknesses": "(-) The results presented in the paper are not entirely convincing. For instance:\n  * In Figure 2, the style editing results between CSD-Edit and Instruct-Pix2Pix + MultiDiffusion are difficult to discern quantitatively. Moreover, noticeable stitching artifacts are visible in the results.\n  * The video editing results available on the webpage exhibit a prominent flickering effect, which detracts from the perceived quality of the output.\n  * In Table 2, the advantages of CSD-Edit over Instruct-NeRF2NeRF in 3D scene editing are marginal, which raises questions about the practical superiority of the proposed method.\n\n(-) The paper does not provide a robustness analysis of the proposed method. It would be beneficial to understand how the method performs under different conditions or with different types of input.\n\n(-) Please refer to the Questions section for more comments.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer qUzm, \n\nWe sincerely appreciate your efforts and comments to improve the manuscript. We respond to your comment in what follows. \n\n---\n\n**[W1] Some of the results which are not entirely satisfactory**\n\nFor your information, we further clarify our experimental results and commit to including these findings in our final manuscript to provide a more comprehensive assessment of our proposed method. \n\n- (Comparison with InstructPix2Pix+MultiDiffusion): The superiority of our method over InstructPix2Pix+MultiDiffusion lies in the controllability over instruction-fidelity across different guidance scales. Please refer to [GR1] in the global response for further details. \n\n- (Regarding the remaining stitching and flickering effects): While our CSD-Edit shows improved consistency compared to prior state-of-the-art baselines across all modalities, this might not be an ultimate approach in ensuring absolute consistency due to the inability of diffusion models and autoencoders. Thus, injecting modality specific data prior could be a promising future work. We will discuss this limitation more prominently in our final manuscript.\n\n- (Quantitative results in 3D scene editing): To address your concerns regarding the quantitative results for 3D scene editing, we conduct additional user studies including other editing tasks. Notably, ours outperforms the baselines by a large margin. Please refer to [GR3] in the global response for further details. \n\n---\n\n\n**[W2 / Q2] Robust analysis of the proposed method / Errors and bias inherited from pre-trained Instruct-Pix2Pix** \n\nHere, we present a detailed explanation on how our method performs on different conditions or types of input. Since we aim to distill the generative prior of InstructPix2Pix, our method performs well at which InstructPix2Pix excels. For instance, our method shows remarkable performance in stylization or changing multiple objects to others. Also, our method is able to do image editing with multiple prompts, i.e., region-wise edit of a panorama image, in Appendix B.1. \n\nBesides, as we mentioned in the limitation section, the errors and biases of InstructPix2Pix might be transferred to our method. In particular, InstructPix2Pix often edits undesirable objects. Therefore, when editing a panorama image without SVGD, we often observe that unwanted objects are changed in some patches, breaking the consistency of the output image. On the other hand, we empirically observe that our mixing scores in CSD acts as a regularizer that prevents abrupt change in images, ensuring better consistency. For instance, see Figure 4 in the supplementary PDF for visual examples: the tiger is generated in the unwanted region of a source image when SVGD is not applied in update. \n\n---\n\n**[Q1] Trade-offs between the quality of the results and required computational resources compared with SDS and SJC**\n \nWe measure the computation time and compare it with the baselines in [GR2] of the global response. \n\n---\n\n**[Q3] Detailed explanation about the number of samples N**\n\nFor clarification, we denote N as total number of images (e.g., total patches of a panorama image or total frames of a video) and we update B minibatch of samples per iteration. Intuitively, using a large B would encourage more consistency among samples, but it is more computationally expensive. Also, we observed that using large batch sizes dilutes the effect of editing. Thus, the batch size controls the tradeoff between computation time, editing quality, and preserving consistency. To further verify our choice of B, we provide additional ablation study on the panorama image editing experiments. Given the same experimental setup as in Section 4.1 with fixed guidance scale=7.5, we swept over B = 4, 8, 12, and measure the CLIP source-target image similarity, CLIP directional image-text similarity, and computation time (iteration per second for total 200 iterations, measured on single A100 40GB GPU). The following table below demonstrates the effect of batch size. \n\n\\begin{array}{lccc}\n\\text{Batch size} & \\text{CLIP Image Sim.} & \\text{CLIP Directional Sim.} & \\text{Time (iter / sec)} \\newline\n\\hline\n\\text{B=4}   & 0.6392 & 0.2401 & 2.86  \\newline\n\\text{B=8}   & 0.6917 & 0.2165 & 1.47  \\newline\n\\text{B=12} & 0.7394 & 0.1953 &  1.02  \\newline\n\\end{array}\n\nAlso, we provide qualitative examples on the effect of batch size in Figure 3 of supplementary PDF. We show that one can control the diversity of generated output, e.g., same penguins are changed to diverse chickens, by choosing appropriate batch size. We will include this ablation study in our final manuscripts. \n\n---\n\n**[Q4 & Q5] View-dependent prompting and handling Janus problem in text-to-3D synthesis via CSD**\n\nIn Figure 6 of our manuscript, we show that CSD enables consistent visual synthesis of 2D images when view-dependent prompting is applied. While these view-dependent prompts might guide the generation of different shapes or contents for each image, CSD helps to generate coherent objects across different views by updating their scores synchronously. Based on this observation, we apply CSD into text-to-3D synthesis by computing CSD on sampled views at each iteration. \n\nThrough experimental results shown in Table 3 and Figure 13 in Appendix, we demonstrate the effectiveness of CSD on text-to-3D generation, especially in potent to mitigate the Janus problem. We believe that the consistent update among views, facilitated by the reduced variance, results in learning better geometry, thus helping better 3D synthesis. Nonetheless, it is important to note that our approach does not entirely remedy the fundamental cause of Janus problem. The fundamental cause of the Janus problem resides in the insufficient understanding of 3D geometry and a lack of diverse views during the training of diffusion models, specifically those views other than front faces. \n\n"}, {"review_id": "EUGLiGVA8I", "reviewer": "Reviewer_j3eS", "summary": "This paper presents a novel method called Collaborative Score Distillation (CSD) for consistent visual synthesis and manipulation. The proposed CSD-Edit utilizes pre-trained text-to-image models and can be competent for panorama image editing, video editing and 3D scene editing tasks, and generate inter-sample consistent results. Sufficient experiments on several tasks have demonstrated that the effectiveness of the proposed method.", "questions": "-\tConsidering the randomness of the results of the generative models, I’m curious about the criteria used to select the results compared in the paper. This detail should be explained for both the baseline methods and the proposed method to eliminate artificially introduced bias.\n-\tRepeatability in the generated/edited images may be worth further analysis and ablation study.\n-\tOther concerns have already been mentioned in Weakness.\n", "limitations": "Yes, the authors have addressed the limitations and potential negative societal impact of their work.", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "-\tThis paper proposes an optimization strategy called Collaborative Score Distillation (CSD) for text-to-image models to perform consistent visual editing, which has demonstrated impressive effectiveness on a variety of tasks including visual editing of panorama images, videos and 3D scenes.\n-\tThe visualization is impressive, qualitatively illustrating the effectiveness of the proposed method and its ability to generate consistent results.\n-\tThe quantitative results also demonstrate the effectiveness of the proposed CSD and its superiority over the existing baseline methods to some extent.\n", "weaknesses": "-\tThe innovation of the proposed Collaborative Score Distillation is somehow limited, such as combining SDS presented in DreamFusion [26] and SVGD to obtain the general form of SDS. \n-\tThe panorama image editing results tend to produce duplicate content, especially in Figure 2 and Figure 10, and it appears that the diversity of images generated by CD-Edit is limited.\n-\tThe improvement of the numerical results of the quantitative comparison cannot solidly support the superiority of the proposed method, because considering the randomness of the generated/edited results of generative models, it may lead to relatively large fluctuations in the numerical results.\n-\tConsidering that the proposed method is an optimization-based method, the time of optimizing a scene should also be taken into account when compared with other baseline methods. The supplementary materials include the number of iterations of implementation details, but there is still no objective time comparison.\n-\tSome metrics are expected to assess the intra-image and inter-image consistency, and if objective metrics are lacking, subjective user study may be taken into consideration.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer j3eS, \n\nWe sincerely appreciate your thoughtful comments, efforts, and time to improve our manuscript. We respond to each of your questions and concerns one-by-one in what follows. Please let us know if you have any comments/concerns that we have not addressed up to your satisfaction.\n\n---\n\n**[W1]  Limited innovation of proposed method** \n\nAs also recognized by Reviewers v8pz and qUzm, we emphasize that our method is not just a naive combination of existing techniques, but rather a novel composition of ideas to address the limitation of SDS in a principled manner. Specifically, we first claim the lack of inter-sample consistency under SDS, which hinders its broader potential applications, e.g., to higher-dimensional visual synthesis. We addressed these challenges by casting them as a multiple-sample variational inference problem and proposed a method that uses SVGD. This reinterpretation serves not only as a simple solution to the problem but also as a practical innovation that is scalable to recent diverse tasks - e.g. adapting text-to-image diffusion models for high-dimensional manipulation. We believe that this could be a useful addition to the field, extending the applicability of SDS for consistent synthesis, which is also highlighted by reviewer v8pz.\n\n---\n\n**[W2 & Q2]  Lack of diversity and repeatability in the edited results when applying CSD-Edit**\n\nFirst, we note that our goal is to generate samples consistently in order to demonstrate the power of our method, which allows synchronous editing across a set of images. Nevertheless, it is possible to control the diversity of the generated image with different mini-batch sizes. Note that using a large minibatch size B would encourage more consistency among samples (see our response to reviewer qUzm [Q3] for more explanation). As shown in Figure 3 in the supplementary PDF file, varying the minibatch size B can control the diversity of the generated output: given similar penguins in the source image, the diverse chickens can be generated by using a small batch size. It is important to note that even with this diversity, coherent structures are preserved. This is because randomly selected samples are processed synchronously during each optimization iteration. Lastly, please refer to [Q3] of response to Reviewer qUzm in additional ablation study on the effect of batch size. \n\n---\n\n**[W3 & Q1]  Consideration of randomness in evaluation**\n\nSince the variance in editing (not generation) tasks from different random seeds is relatively small, we did not highlight the randomness in the evaluation. Instead, we ensured that the hyperparameters were fine-tuned for baseline methods for fair comparison. However, to address your concern, we conducted an additional evaluation with the randomness of generation models into consideration. We repeated under the identical experimental setup using 5 different random seeds and reported average scores along with standard deviations for each method to evaluate desired edits. As demonstrated in Table 1 and Table 2 of the supplementary pdf, CSD-Edit consistently outperforms the baselines, highlighting the robustness of our approach across different runs in achieving desired edits. \n\n---\n\n**[W4]  Lack of comparison of computation time**\n\nTo address your concern, we measure the computation time and compare with the baselines. Please refer to [GR2] in the common response for further details.\n\n---\n\n**[W5]  Lack of subjective user study**\n\nIn addition to our objective evaluation, we conduct additional user studies to compare with video editing and 3D scene editing baselines. Notably, ours outperforms the baselines by a large margin. Please refer to [GR3] in the common response.\n"}, {"review_id": "v5GJP0VtSz", "reviewer": "Reviewer_yGH2", "summary": "The paper introduces an approach to achieve consistent visual editing by leveraging a pre-trained pix2pix diffusion model. The authors propose a generalization of the SDS loss (originating from DreamFusion) to a CSD loss, which utilizes Stein variational gradient descent. This new loss function enables the joint distillation of multiple samples from a text-to-image diffusion model. The CSD loss is applicable to diverse visual editing scenarios, including panorama images, videos, and 3D scenes. Additionally, the CSD loss can be utilized for text-to-3D generation as well.", "questions": "- What is the reason behind the focus on editing applications in most of the experiments? Was panorama generation also explored by the authors?\n- How does the paper provide an explanation for the observed comparison in the panorama editing results shown in figure 2? Was there an investigation into the potential use of higher weighting parameters for the baselines or other configurations?\n- The results obtained for text-to-3D generation appear to be quite similar to the results achieved with DreamFusion using the SDS loss. Can the authors offer an explanation for this similarity in performance?", "limitations": "The authors discussed limitation, however the limitation figure is shown soley in the appendix. Morover, the authors did not discussed training times which could serves as on of the limitations.", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The authors provided results for various visual editing applications, such as panorama images, videos, and 3D scenes, as well as text-to-3D generation. This demonstrates the promise of the suggested method's flexibility for numerous applications.\n- The paper reads well and flows smoothly.\n- The authors presented all the required preliminaries, making the paper self-contained for newcomers.", "weaknesses": "- The paper lacks reproducibility due to several crucial missing details:\n  - The paper does not specify the meaning or selection process of parameter N in all applications, nor how to choose the final parameter theta from the set of N parameters.\n  - The formulation is missing information on the aggregation over multiple views/frames/crops, making it unclear how to implement this aspect.\n  - The explicit specifications of classifier-free guidance weights are missing, particularly in the 3D generation application where DreamFusion originally requires a high weight to produce desired results.\n- I don't quite undertsand why the additional epsilon is  required in both equations 8 and 9, where from equation 7 it is clear that only the score (epsilon_phi) is given. I find this to be very not justified, nor the transformation from random noise to the image-conditioned prediction in equation 9. It seems like there were several options to take here instead as well, and the specific choice seems arbitrary. The provided ablation in section 4.4 is insufficient to justify these choices, as it does not cover all possible alternatives (such as using only the score or other deterministic predicted noise).\n- The paper lacks preliminary information on multidiffusion, despite being depicted in figure 2. The related work section is generally lacking and should elaborate on other methods beyond DreamFusion.\n- Since the suggested method is presented as a generalization of the SDS loss, a more extensive comparison to SDS should have been provided in all applications. This is an important baseline that is currently missing.\n- Figure 1 is overcrowded and confusing. It would be clearer if the generated panorama on the left side corresponded to the text description rather than being identical to the source image.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear reviewer yGH2, \n\nWe sincerely appreciate your efforts and comments to improve the manuscript. We respond to your comment in what follows. \n\n---\n\n**[W1]  Lack of detailed explanation for reproducibility**\n\nAs for reproducibility, we comprehensively included the details required to implement our method, both in the main manuscript and appendix:\nTo clarify, N denotes the total number of images and we select B minibatch of samples at each iteration to update. The implementation details are in Appendix D. Also, we provide additional ablation studies on the effect of batch size B in response [Q3] of Reviewer qUzm.\nImplementation details regarding aggregation can be found in Section 3.3 and Appendix D. \nDetails on classifier-free guidance weights can be found in Appendix D for text-guided editing tasks, and in Appendix B.2 for text-to-3D generation\n\nFor further clarification, we offer line-by-line explanations alongside our submitted code for video editing. The submitted code shows that B is determined by the number of video frames, i.e.,B = N (line 132), how the final edited images are obtained (line 247), and where the combination of scores from multiple samples for gradient updates occurs (lines 227 and 229).\n\n\nLastly, we note that the code has been submitted as part of the supplementary material. We advocate for reproducible research and will open-source our code that reproduces results in the paper.\n\n---\n\n**[W2]  Detailed analysis about the choice of subtracted baseline noise**\n\nOur rationale behind the image-conditional noise is elaborated in Appendix A, as well as in Sections 3.2 and Section 4.4. Furthermore, we elucidate the choice of using image-conditioned noise as follows:\n\n- Subtraction of random noise: Directly applying Eq. (7) solely with the score function of the target distribution leads to a high variance of gradient, which can severely hinder convergence. Thus, as also demonstrated in DreamFusion, subtracting random noise is crucial as an effective regularization. \n- Introduction of image-conditional noise: However, as shown in Figures 7 and 12 in our manuscript, the default choice of SDS results in severely blurred outputs, as the noise-denoise process of SDS blurs the image. Thus, we propose to subtract image-conditional noise so that the diffusion noise only alters the part where the text instructs to change. This approach is supported by the principle of Wasserstein gradient flow, where the optimal gradient flow in variational inference is given as the difference between the target score function and source score function (See response [Q1] to Reviewer v8pz for further details). \n- Other choices for baseline noise: Recent works show different choices in subtracting baseline noise: Delta Denoising Score [1] estimates the noise of the source image by providing a suitable source prompt, and ProlificDreamer [2] fine-tunes U-Net to obtain the noise of the source distribution (Also, refer to the response [Q1] to Reviewer v8pz). However, those are not favorable in our cases as source prompts are not given in real image-editing nor fine-tuning U-Net is computationally expensive. \n\n---\n\n**[W3]  Missing information about detailed explanation of baseline methods**\n\nIn response, we will elaborate more details about baseline methods and clarify the difference from our method in our final manuscript for a comprehensive understanding.\n\n---\n**[W4] Lack of extensive comparison with SDS in visual editing experiments**\n\nWhile SDS is the most relevant approach to our method, the quality of SDS edited images is superseded by more recent works, which we considered as baselines for comparison, as explained in [W2]. Thus, we primarily compared our method with much stronger, state-of-the-art baselines of each modality to show its remarkable performance. Nevertheless, we compared our method with SDS in ablation studies, as shown in Figure 7, and Appendix C (random noise stands in for SDS), and qualitative examples are in Figure 12. We will clarify this more explicitly in our final manuscript.\n\n---\n**[W5]  Concept figure is confusing**\n\nThank you for the detailed suggestions. We revised Figure 1 in supplementary PDF to better illustrate our method in overview.\n\n---\n**[Q1]  Reasons behind the focus on editing applications rather than generation**\n\nWe primarily focused on editing applications in most of our experiments due to the inherent limitations associated with SDS. Specifically, SDS tends to converge towards certain modes, resulting in blurred outputs with fewer details. Given this issue, we have opted to focus on text-guided editing of panorama images utilizing image-conditioned noise, rather than generation using random noise. However, we think extending our idea for generation tasks would be an interesting future work to explore.\n\n---\n**[Q2]  More detailed explanation for the results shown in Figure 2**\n\nRegarding additional explanation for the panorama image experiments, we refer to [GR1] in the common response. Our intention in Figure 2 was to emphasize that our method has more controllability compared to InstructPix2Pix+MultiDiffusion by varying the guidance scale when edit images.\n\n---\n\n**[Q3]  Reason for the similarly obtained results between SDS and CSD for text-to-3D synthesis**\n\nThe reason for the similar results is because of the identical experimental setup (e.g., hyperparameters for NeRF training, see Appendix B.2), especially using the same random seeds. Nonetheless, CSD presents finer details compared to SDS, as shown in Table 3 and Figure 13 in the appendix, illustrating the qualitative benefits of CSD.\n\n---\n\n**[L1]  Lack of comparison of computation time**\nFollowing your suggestion, we measure the computation time compared to the baselines in [GR2] in the global response.\n\n---\n\n**Reference**\n\n[1] Hertz et al., Delta Denoising Score, ICCV 2023\n\n[2] Wang et al., ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation, arXiv 2023\n\n"}]

Author rebuttal:
Dear reviewers and AC,

We sincerely appreciate your valuable time and effort spent reviewing our manuscript.

As reviewers highlighted, we believe our paper presents a principled and novel method (v8pz, qUzm) that performs effective visual editing of versatile modalities (v8pz, qUzm, j3es, yGH2), validated by extensive experiments in both quantitative and qualitatively (v8pz, qUzm, j3es), followed by comprehensive presentation (v8pz, qUzm, yGH2).

Here, we collected the common questions that multiple reviewers have asked and responded to each question one-by-one in what follows. We also kindly ask you to check out the attached supplementary PDF file together. Please let us know if you have any comments/concerns that we have not addressed up to your satisfaction.

---

**[GR1] More detailed explanation for the results shown in Figure 2 (Reviewer qUzm, yGH2)**

Figure 2 illustrates two benefits of our method: spatial consistency and instruction-fidelity. Patch-wise editing of a panorama image results in spatial inconsistencies due to visible patch boundaries (Figure 2, right top). This is mitigated in InstructPix2Pix+MultiDiffusion (Figure 2, middle row) by using overlapping patches, but the edited image loses its fidelity to the instruction as the scores are diluted by other scores, e.g., one patch may respond to the instruction much more or much less compared to others, thus the effect is diluted in such cases. On the other hand, CSD is able to mitigate such a diluting effect by optimizing with a subset of images. Thus, in Figure 2, given the same guidance scale, our method shows better fidelity to the instruction.  

The effect of guidance scales on the instruction-fidelity of image editing is demonstrated in Figure 5 of our manuscript. Each dot on the graph represents different guidance scales, and a direct comparison of each guidance scale shows a noticeable gap between InstructPix2Pix+MultiDiffusion and CSD-Edit in terms of CLIP directional scores. This underscores the superiority of our method in balancing source-target image consistency and instruction-fidelity across scales, thereby highlighting its controllability, particularly given the subjective nature of achieving a desired edit. In the enhancement of representation, we revise Figure 5 (now Figure 2 in the supplementary PDF) following the reviewers’ comments and will include the revised figure along with a detailed explanation in our final manuscript. 

***
**[GR2] Measurement and comparison of computation time (Reviewer qUzm, j3eS, yGH2)**

Regarding the computational efficiency of our method, we measure the computation times and compare them with the baselines of each task. All these evaluations were conducted on a single NVIDIA A100 80GB and AMD EPYC 7V13 64-Core Processor. 

- For panorama image editing experiments, we compare with the baseline InstructPix2Pix+MultiDiffusion. Note that the computation time of both methods depends on the input image resolution, whereas we show that our method becomes more efficient as the resolution goes higher. The following table below shows how the computation time (total time in seconds) differs by the size of the input image. 
 \begin{array}{lccc}
\text{Method} & \text{Resolution} & \text{Total time (sec.)} \newline 
\hline
 \text{InstructPix2Pix+MultiDiffusion} & 1920\times640 & 62  \newline
\text{CSD-Edit (Ours)} & 1920\times640 & 68  \newline
\hline
 \text{InstructPix2Pix+MultiDiffusion} & 3968\times 4352  & 487  \newline
\text{CSD-Edit (Ours)} & 3968\times 4352  & 275   \newline
\end{array}

  Note that the baseline method requires computing noise estimates of every patch at each diffusion step, while our method only requires computing a minibatch of patches per iteration.

- In video editing experiments, we measure the total computation time of our method and baseline methods to obtain the results shown in Figure 11. The results are shown in the following table below: 
\begin{array}{lc}
\text{Method} & \text{Total time (sec.)} \newline 
\hline
\text{FateZero}   & 192 \newline
\text{Pix2Video}   & 77  \newline
\text{CSD-Edit (Ours)}   & 423  \newline
\end{array}

  Although our method requires more computation time compared to the baselines, users can choose to early stop the optimization when they achieve a desired edit, which is not an attribute of the baseline methods as they resort to diffusion samplers.

- In text-to-3D generation experiments, when directly comparing with DreamFusion (or SJC), there is a slight increase in the computational cost due to the usage of LPIPS metric as distance for RBF kernel. For instance, it takes 1 hour to generate a 3D model with DreamFusion, while using CSD takes 84 minutes. We note that the increment is not substantial, however, ours yields better qualities with finer details, as shown in Table 3 and Figure 13 in the Appendix of our manuscript. 

We will add this information to our final manuscript.

---

**[GR3] User study (Reviwer qUzm, j3eS)**

While we have primarily relied on objective metrics for assessing consistency and instruction-fidelity, we agree that a subjective user study could provide valuable insights, especially given the subjective nature of editing tasks. Thus, we conducted additional user studies, where we asked three questions in evaluating the editing methods: the consistency of the edited results, frame-wise image instruction-fidelity, and the editing quality.  For each of the three studies, we asked 20 subjects to rank different methods. As shown in Table 3 and Table 4 in the supplementary PDF file, our method consistently outperforms others achieving the best user preferences across all three aspects. We commit to including these findings in the final version of our manuscript to provide a more comprehensive assessment of our proposed method.




Meta-review: This paper proposes a collaborative score distillation technique for visual editing, including panorama image, video, and 3D. In addition to editing, the authors also show the results of text-to-3D generation. After the rebuttal, most reviewer concerns are addressed by the authors. However, the reviewers keep the concern about the novelty and the scope of this paper:
1. Leveraging SVGD for the task seems somewhat incremental;
2. It is unclear if the goal is to improve general SDS method, if so, there is no major comparisons on the generation side;
3. All the experiments should compare the proposed approach with the original SDS approach.

After going over the literature and the comments made by the reviewers and the authors, the AC decide to accept the paper as:
1. Many reviewers still hold positive opinions although the novelty may be somewhat incremental.
2. In the rebuttal, the authors specify that the scope of this paper is to propose an effective approach for visual editing, not improving general SDS.
3. The authors do show the comparison to the original SDS approach in some experiments, and in the other experiments, the authors present the results of stronger baselines.

Despite acceptance, the AC sincerely **ask** the authors to
1. Include all the discussions/experimental results in the rebuttal to the paper or supplementary materials.
2. Revise the paper (please consider revising the title as well) to ensure that **the manuscript describes the proper scope**. Specifically, as discussed during the rebuttal, the authors should be responsible to make sure the reader understand that the goal is for "editing" not general "synthesis", e.g., the title is not accurate from this perspective.
3. The authors should include the results of vanilla SDS in all the other experiments presented in the paper.
Decision: Accept (poster)