Score Distillation via Reparametrized DDIM

Authors: artem  lukoianov
haitz  sáez de ocáriz borde
kristjan  greenewald
vitor campagnolo guizilini
timur  bagautdinov
vincent  sitzmann
justin  solomon
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-10 18:52:36

Abstract: While 2D diffusion models generate realistic, high-detail images, 3D shape generation methods like Score Distillation Sampling (SDS)  built on these 2D diffusion models  produce cartoon-like, over-smoothed shapes. To help explain this discrepancy, we show that the image guidance used in Score Distillation can be understood as the velocity field of a 2D denoising generative process, up to the choice of a noise term. In particular, after a change of variables, SDS resembles a high-variance version of Denoising Diffusion Implicit Models (DDIM) with a differently-sampled noise term: SDS introduces noise i.i.d. randomly at each step, while DDIM infers it from the previous noise predictions. This excessive variance can lead to over-smoothing and unrealistic outputs. We show that a better noise approximation can be recovered by inverting DDIM in each SDS update step. This modification makes SDS's generative process for 2D images almost identical to DDIM. In 3D, it removes over-smoothing, preserves higher-frequency detail, and brings the generation quality closer to that of 2D samplers. Experimentally, our method achieves better or similar 3D generation quality compared to other state-of-the-art Score Distillation methods, all without training additional neural networks or multi-view supervision, and providing useful insights into relationship between 2D and 3D asset generation with diffusion models.

Main paper: 
 introduction: 
 Image generative modeling saw drastic quality improvement with the advent of text-to-image diffusion models [1] trained on billion-scale datasets [2] with large parameter counts [3] . From a short prompt, these models generate photorealistic images, with strong zero-shot generalization to new classes [4] . Efficient training methods for image data, combined with Internet-scale datasets, enabled the development of these models. However, applying similar techniques to domains where huge datasets are scarce, such as 3D shape generation, remains challenging.The need for 3D objects in downstream applications like vision, graphics, and robotics motivated methods like Score Distillation Sampling (SDS) [5] and Score Jacobian Chaining (SJC) [6] , which optimize volumetric 3D representations [7, 8, 9] using queries to a 2D generative model [10] . In every iteration, SDS renders the current state of the 3D representation from a random viewpoint, adds noise to the result, and then denoises it using the pre-trained 2D diffusion model conditioned on a text prompt. The difference between the added and predicted noise is used as a gradient-style update on the rendered images, which is propagated to the parameters of the 3D model. The underlying 3D representation helps make the generated images multi-view consistent, and the 2D model guides individual views towards a learned distribution of realistic images.In practice, however, as noted in [11, 12, 13] , SDS often produces 3D representations with oversaturated colors and over-smoothed textures (fig. 1d ), not matching the quality of the underlying 2D model. Existing approaches tackling this problem improve quality at the cost of expensive re-training or fine-tuning of the image diffusion model [11] , complex multi-stage handling of 3D representations like mesh extraction and texture fine-tuning [14, 15, 11] , or altering the SDS guidance [13, 12, 16] .As an alternative to engineering-based improvements to SDS, in this paper we reanalyze the vanilla SDS algorithm to understand the underlying source of artifacts. Our key insight is that the SDS update rule steps along an approximation of the DDIM velocity field. In particular, we derive Score Distillation from DDIM with a change of variables to the space of single-step denoised images. In this light, SDS updates are nearly identical to DDIM updates, apart from one difference: while DDIM samples noise conditionally on the previous predictions, SDS resamples noise i.i.d. in every iteration. This breaks the denoising trajectory for each independent view and introduces excessive variance. Our perspective unifies DDIM and SDS and helps explain why SDS can produce blurry and over-saturated results: the variance-boosting effect of noisy guidance is usually mitigated with high Classifier-Free Guidance (CFG) [17] to reduce sample diversity at the cost of over-saturation [18] .Based on our analysis, we propose an alternative score distillation algorithm dubbed Score Distillation via Inversion (SDI), closing the gap to DDIM. We obtain the conditional noise required for consistency of the denoising trajectories by inverting DDIM on each step of score distillation (fig. 5 ). This modification yields 3D objects with high-quality textures consistent with the 2D diffusion model (fig. 1e ). Moreover, in 2D, our method closely approximates DDIM while preserving the incremental generation schedule of SDS (fig. 1c ).Our key contributions are as follows:• We prove that guidance for each view in the SDS algorithm is a simplified reparameterization of DDIM sampling: vanilla SDS samples random noise at each step, while DDIM keeps the trajectories consistent with previously-predicted noise. • We propose a new method titled Score Distillation via Inversion (SDI), which replaces the problematic random noise sampling in SDS with prompt-conditioned DDIM inversion and significantly improves 3D generation, closing the quality gap to samples from the 2D model. • We systematically compare SDI with the state-of-the-art Score Distillation algorithms and show that SDI achieves similar or better generation quality, while not requiring training additional neural networks or multiple generation stages.2 Related work 3D generation by training on multi-view data. Recent 3D generation methods leverage multi-view or 3D data. Zero123 [19] and MVDream [20] generate consistent multi-view images from text; a 3D radiance field is then obtained via score distillation. Video generative models can be fine-tuned on videos of camera tracks around 3D objects, similarly yielding a model that samples multi-view consistent images to train a 3D radiance field [21, 22] . Diffusion with Forward Models [23] and Viewset Diffusion [24] directly train 3D generative models from 2D images. While these methods excel at generating multi-view consistent, plausible 3D objects, they depend on multi-view data with known camera trajectories, limiting them to synthetic or small bundle-adjusted 3D datasets. We instead focus on methods that require only single-view training images.Distilling 2D into 3D. Score Distillation was introduced concurrently in Dreamfusion or SDS [5] , Score Jacobian Chaining (SJC) [6] , and Magic3D [14] . The key idea is to use a frozen diffusion model trained on 2D images and "distill" it into 3D assets. A volumetric representation of the shape is rendered from a random view, perturbed with random noise, and denoised with the diffusion model; the difference between added and predicted noise is used to improve the rendering. These works, however, suffer from over-smoothing and lack of detail. Usually a high value of classifier free guidance (CFG ≈ 100) [17] is used to reduce variance at the cost of over-saturation [18] .ProlificDreamer [11] generates sharp, detailed results with standard CFG values (≈ 7.5) and without over-saturation. The key idea is to overfit an additional diffusion model to specifically denoise the current 3D shape estimate. Fine-tuning the second model, however, is cumbersome and theoretical justification for this change is still unclear. Recent papers further improve on ProlificDreamer's results or try to explain its behavior. SteinDreamer [25] , for example, hypothesizes that ProlificDreamer's improvements come from variance reduction of the sampling procedure [26] .Other papers propose heuristics that improve SDS. [12, 16, 27] decompose the guidance terms and speculate about their relative importance. Empirically, visual quality can be improved by suppressing the denoising term with negative prompts [12] or highlighting the classification term [16] . [14, 15, 11, 28] use multi-stage optimization: they first train a volumetric representation and then extract a mesh or voxel grid to fine-tune geometry and texture. HiFA [13] combines ad-hoc techniques: additional supervision in latent space, time annealing, kernel smoothing, z-variance regularization, and a pretrained mono-depth estimation network to improve the quality of single-stage NeRF generation.LucidDreamer, or Interval Score Matching (ISM), hypothesizes based on empirical evidence that in SDS, the high-variance random noise term and large reconstruction error of a single-step prediction togehter cause over-smoothing [29] . Based on these observations, the authors replace the random noise term in SDS with noise obtained by running DDIM inversion and introduce multi-step denoising to improve reconstruction quality. As we will show in section 4, the update rule of ISM can be seen as a special case of our formulation. Moreover, our analysis reveals that the added noise should be inferred conditionally on the text prompt y, which further improves quality.In this work, rather than augmenting the SDS pipeline or relying on heuristics, we derive Score Distillation through the denoising process of DDIM and propose a simple modification of SDS that significantly improves 3D generation.3 Background Diffusion models. Denoising Diffusion Implicit Models (DDIM) generate images by reversing a diffusion process [30, 31, 32] . After training a denoiser ϵ t θ and freezing its weights θ, the denoising process can be seen as an ODE on rescaled noisy images x(t) = x(t)/ α(t). Given a prompt y and current time step t ∈ [0, 1], the denoising process satisfies:EQUATIONwhere x( 1) is sampled from a Gaussian distribution, σ(t) = 1 -α(t)/ α(t), and α(t) are scaling factors. When discretized with forward Euler, this equation yields the following update to transition from step t to a less noisy step t -τ < t:EQUATIONThe DDIM ODE can also be integrated in reverse direction to estimate x(t) for any t ∈ [0, 1] from a clean image x 0 . This operation is called DDIM inversion and is studied in multiple works [33, 34] .Classifier-free guidance. Classifier-free guidance (CFG) [17] provides high-quality conditional samples without gradients from auxilary models [35] . CFG modifies the noise prediction εt θ (score function) by linearly combining conditional and unconditional predictions:EQUATIONwhere the guidance scale γ is a scalar, with γ = 0 corresponding to unconditional sampling and γ = 1 to conditional. In practice, larger values γ > 1 are necessary to obtain high-quality samples at a cost of reduced diversity and extreme over-saturation. We demonstrate the effect of different CFG values in fig. 3 . For the rest of the paper we use the modified CFG version of the denoiser ϵ t θ x(t), y . Score Distillation. Diffusion models efficiently generate images and can learn to represent common objects from arbitrary angles [36] and with varying lighting [5] . Capitalizing on this success, Score Distillation Sampling (SDS) [5] distills a pre-trained diffusion model ϵ t θ to produce a 3D asset. In practice, the 3D shape is usually parameterized by a NeRF [7] , InstantNGP [8] , or Gaussian Splatting [9] . Multiple works additionally extract an explicit representation for further optimization [14, 15, 11, 28] . We use InstantNGP [8] to balance between speed and ease of optimization.Denote the parameters of a differentiable 3D shape representation by ψ ∈ R d and differentiable rendering by a function g(ψ, c) : R d × C → R N ×N that returns an image given camera parameters c ∈ C. Intuitively, in each iteration, SDS samples c, renders the corresponding (image) view g(ψ, c), perturbs it with ϵ ∼ N (0, I) to level t ∼ [0, 1], and denoises it with ϵ t θ ; the difference between the true and predicted noise is propagated to the parameters of the 3D shape. More formally, after sampling the camera view c and randomly drawing a time t, SDS renders the volume and adds Right: Each step of DDIM steps toward a denoised image. This can be seen as a step to x 0 (t) and a step back to a slightly less noisy image. Through a change-of-variables we obtain a process on x 0 (t).Gaussian noise ϵ to obtain a noisy image x(t) = α(t)g(ψ, c) + 1 -α(t)ϵ. Then, SDS improves the generated volume by using a gradient(-like) direction to update its parameters ψ:EQUATION)We refer to the term ϵ t θ x(t), y -ϵ as guidance in score distillation, as it 'guides' the views of the shape. In theory, this expression may not correspond to the true gradient of a function and there are many hypotheses about its effectiveness [5, 25, 12, 16, 26] . In this work, we show that instead it can be seen as a high-variance version of DDIM after a change of variables.4 Linking SDS to DDIM Discrepancy in image sampling. Beyond the lack of formal justification of eq. ( 4), in practice SDS results are over-saturated and miss details for high CFG values, while they are blurry for low CFG values. To illustrate this phenomenon, fig. 1 shows a simple experiment, inspired by [11] : We replace the volumetric representation in eq. ( 4) with an image g 2D (ψ 2D , c) := ψ 2D ∈ R N ×N . In this case, SDS becomes an image generation algorithm that can be compared to other sampling algorithms like DDIM [37] . Even in this 2D setting, SDS fails to generate sharp details, while DDIM with the same underlying diffusion model produces photorealistic results, motivating our derivation below.Why not use DDIM as guidance? Given the experiment above, a natural question to ask is if it is possible to directly use DDIM's update direction from eq. (1) as SDS guidance in eq. ( 4) to update the 3D representation. The problem with this approach lies in the discrepancy between the training data of the denoising model and the images generated by rendering the current 3D representation. More specifically, the denoising network expects an image with a certain level of noise corresponding to time t as defined by the forward (noising) diffusion process, whereas renderings of 3D representations g(ψ, c) evolve from a blurry cloud to a well-defined sample (fig. 4 Evolution of x 0 (t). Instead of seeing DDIM as a denoising process defined on the space of noisy images x(t), we reparametrize it to a new variable:EQUATIONIn words, x 0 (t) is the noisy image at time t denoised with a single step of noise prediction. Empirically, the evolution of x 0 (t) is similar to the evolution of g(ψ, c)-from blurry to sharp. The left side of fig. 4 compares these processes. This similarity motivates us to rewrite eq. (1) in terms of x 0 (t), and to understand SDS as applying similar updates to the renderings of its 3D representation.Reparametrizing DDIM. Figure 4 (right) shows schematically how the DDIM update to x(t) alternates between denoising to obtain x 0 (t) and adding the predicted noise back to get a cleaner x(t -τ ). Based on the intuition above, we reorder the steps, adding noise to x 0 (t) and then denoising to estimate x 0 (t -τ ). Consider neighboring time points t and t -τ < t in discretized DDIM eq. ( 2) (lower time means less noise). We rewrite eq. ( 2) using the definition of x 0 (t) from eq. ( 5) to find which is consistent with the intuition behind SDS: improving an image involves perturbing the current image and then denoising it with a better noise estimate. We cannot directly apply eq. ( 6) to SDS in 3D, since it depends on x(t); if we think of x 0 (t) as similar to a rendering of the 3D representation for some camera angle, it is unclear how to obtain a consistent set of preimages x(t) at each step of 3D generation. From eq. ( 5), however, x(t) should satisfy the following fixed point equation:EQUATIONEQUATIONor rewritten in terms of noise ϵ = [x(t) -α(t)x 0 (t)]/ 1 -α(t):EQUATIONDefine κ t y x 0 (t) = ϵ as a solution of this equation given x 0 (t). Then, we can write:EQUATIONThus, eq. ( 6) turns into: x 0 (t-τ ) = x 0 (t)-σ(t-τ ) ϵ t-We can already see that the structure of eq. ( 10) is very similar to the SDS update rule in eq. ( 4). Note that the update direction in eq. ( 10) is the same as in the SDS update rule in eq. ( 4), where κ t y plays the role of the random noise sample ϵ. We could use it as a guidance for the 3D generative process in SDS by replacing ϵ in eq. ( 4) with κ t y (x 0 (t)). In practice, however, it is hard to solve eq. ( 8), as ϵ t θ is high-dimensional and nonlinear. In an unconstrained 2D generation, κ t y can be cached from a previous denoising step, matching the update step to DDIM exactly as in fig. 1c . In 3D, however, this is impossible due to the simultaneous optimization of multiple views and projections to the space of viable 3D shapes. Below we show that a naïve approximation replacing κ t y with a Gaussian yields SDS, and we will propose alternatives that are more faithful to the derivation above.SDS as a special case. From eq. ( 10), to get a cleaner image, we need to bring the current image to time t with noise sample κ t y , denoise the obtained image, and then subtract the difference between added and predicted noise from the initial image. A coarse approximation of κ t y uses i.i.d. random noise κ SDS (x 0 (t)) ∼ N (0, I), matching the forward process by which diffusion adds noise. This choice of κ SDS precisely matches the update rule eq. ( 10) to the SDS guidance in eq. ( 4).ISM as a special case. The main update formula in Interval Score Matching (ISM) [29] is a particular case of eq. ( 10), where κ t y is obtained via DDIM inversion without conditioning on the text prompt y. As demonstrated in section 5, DDIM inversion approximates a solution of eq. ( 10), explaining the improved performance of ISM. However, our analysis suggests that an even better-performing κ t y should incorporate the text prompt y, which further improves the results and avoids over-saturation. 
experimental result reproducibility: 
 Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)? Answer: [Yes] Justification: Our method is described fully and relevant experimental details are clearly provided. Generation examples are not cherry picked unless explicitly stated. Additionally, we provide the code of our algorithm in supplementary and plan to make it public upon acceptance. Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. 
score distillation via inversion (sdi): 
 As we have shown, SDS follows the velocity field of reparametrized DDIM in eq. ( 10), when κ SDS (x 0 (t)) is randomly sampled in each step. Our derivation, however, suggests that κ SDS (x 0 (t)) could be improved by bringing it closer to a solution of the fixed-point equation in eq. ( 8). Indeed, randomly sampling κ SDS as in Dreamfusion yields excessive variance and blurry results for standard CFG values, while using higher CFG values leads to over-saturation and lack of detail. On the other hand, solving eq. ( 8) exactly is challenging due to its high dimensionality and nonlinearity.Like ISM [29] , we suggest to obtain κ t y by inverting DDIM, that is, by integrating the ODE in eq. ( 1) with t evolving backwards (from images to noise) as in [37, 33, 34] . As we can see in eq. ( 8), κ t y should be a function of the text prompt y, leading us to perform DDIM inversion conditionally on y, unlike ISM. This process approximates but is not identical to the exact solution for κ t y : fixed points of eq. ( 8) invert a single large step of DDIM, while running the ODE in reverse inverts the entire DDIM trajectory. We ablate alternative choices for κ t y in section 6.2 and conclude that in practice DDIM inversion offers the best approximation quality. Additionally, to match the iterative nature of DDIM, we employ a linear annealing schedule of t. We refer to our modified version of SDS as Score Distillation via Inversion, or SDI. Starting with a rendering of a 3D shape we demonstrate how each algorithm perturbs it (x(t) variable on the top row) and how it is denoised with a single step of diffusion (x 0 (t) variable on the bottom row). The prompt used is "Pumpkin head zombie, skinny, highly detailed, photorealistic, side view."Figure 6 shows the effect of inferring the noise via DDIM inversion instead of sampling it randomly. The special structure of the improved κ t y results in more consistent single-step generations and produces intricate features at earlier times. When inverted and not sampled, the noise appears 'in the right place': in SDS the noise covers the whole view, including the background, whereas in ours the noise is concentrated on the meaningful part of the 3D shape. This improves geometric and temporal coherency for x 0 (t) predictions even for large t. The reduced variance drastically increases sharpness and level of detail. Moreover, it allows to reduce CFG value of generation γ fwd to the standard 7.5, avoiding over-saturation. Another interesting finding is that DDIM inversion works best when the reverse integration is performed with negative CFG γ inv = -γ fwd = -7.5. The overview of our method is presented in fig. 5 , the details about inversion algorithm are presented in section 6.2, and the implementation details are discussed in appendix A.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [Yes] Justification: Code is included in the supplementary material. No data is needed as we generate all results from prompting a preexisting diffusion model which we specify in the experiment section. Guidelines:• The answer NA means that paper does not include experiments requiring code.• Please see the NeurIPS code and data submission guidelines (https://nips.cc/public/ guides/CodeSubmissionPolicy) for more details. • While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark). While this is a very real problem in the industry at this time, in our view, our work represents only a marginal (if any) increase in such risks, as highly convincing deepfake tools are already widely available. We discuss this in the conclusion of the paper. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
3d generation: 
 We demonstrate the high-fidelity 3D shapes generated with our algorithm in fig. 2 and  provide more examples of 360 • views in appendix H.2. A more detailed qualitative and quantitative comparison of our method with ISM [29] is provided in appendix B. Additionally, we report the diversity of the generated shapes in appendix C.Qualitative comparisons. Figure 7 compares 3D generation quality with the results reported in past work using a similar protocol to [11, 12] . For the baselines we chose: Dreamfusion [5] (the work we build on), Noise Free Score Distillation (NFSD) [12] (uses negative prompts in SDS), ProlificDreamer [11] (fine-tunes and trains a neural network to denoise the 3D shape), Interval Score Matching (ISM) [29] (obtain the noise sample by inverting DDIM and perform multi-step denoising), and HiFA [13] (guides in both image and latent spaces, regularizes the NeRF, and supervises the geometry with mono-depth estimation). Our figures indicate that Score Distillation via Inversion (SDI) yields similar or better results compared with state-of-the-art. Appendix H.3 presents more extensive comparison. Quantitative comparison. We follow [5, 16, 25] to quantitatively evaluate generation quality. Table 1 provides CLIP scores [38] to measure prompt-generation alignment, computed with torchmetrics [39] and the ViT-B/32 model [40] . We also report ImageReward (IR) [41] to imitate possible human preference. We include CLIP Image Quality Assessment (IQA) [42] to measure quality ("Good photo" vs. "Bad photo"), sharpness ("Sharp photo" vs. "Blurry photo"), and realism ("Real photo" vs. "Abstract photo"). For each method, we test 43 prompts with 50 views. For multi-stage baselines, we run only the first stage for fair comparison. We report the percentage of generations that run out-of-memory or generate an empty volume as diverged ("Div." in the table). as well as mean run time and VRAM usage. For VRAM, we average the maximum usage of GPU memory between runs. As many baselines are not open-source, we use their implementations in threestudio [43] . SDI outperforms SDS and matches or outperforms the quality of state-of-the-art methods, offering a simple fix to SDS without additional supervision or multi-stage training. Proposed improvements. Figure 8 ablates the changes we implement on top of SDS. Starting from Dreamfusion [5] with CFG 7.5 we incrementally add: higher NeRF rendering resolution (64 × 64 to 512 × 512), linear schedule on t, and-our core contribution-DDIM inversion. The results clearly demonstrate that the main improvement in quality comes from the inferred noise.Choice of κ t y (x). The key component of our algorithm is the choice of inferred noise κ t y (x). In theory, κ t y (x) should solve eq. ( 8), but it is impractical to do so. Hence, in fig. 9 we compare the following choices of κ t y (x) and report their numerical errors:• Random, resampled: Sample κ t y (x) in each new update step from N (0, I) ; • Random, fixed: Sample κ t y (x) from N (0, I) once, and keep it fixed for each iteration; • Fixed point iteration: Since the optimal solution is a fixed point of eq. ( 8), initialize κ t y (x) ∼ N (0, I) and run fixed point iteration [45] for 10 steps (in practice, more steps did not help).• SGD optimization: Optimize κ t y (x) via gradient decent for 10 steps, initializing with noise. • DDIM inversion: Run DDIM inversion for int(10t) (fewer steps for smaller t) steps to time t, with negative CFG γ inv = -7.5 for inversion and positive γ fwd = 7.5 for forward inference. The left side of fig. 9 compares the choices for 3D generation qualitatively; the right side plots error induced in eq. ( 8) (rescaled to x 0 variable due to its ambiguity around 0). As can be seen, the regressed noise has a big impact on the final generations. Both fixed point iteration and optimization via gradient descent fail to improve the approximation of κ t y , while fixed point iteration diverges in 3D. On the other hand, DDIM inversion yields a reasonable approximation of κ t y and significantly improves 3D generation quality. We provide visual comparison of the obtained noisy images for each baseline in appendix D. y choices in our algorithm. We show the obtained 3D generations on the left (Fixed Point Iteration diverges), and the numerical error in eq. ( 8) induced for each timestep on the right. The resampled and fixed noise strategies produce the same error. CFG for inversion. [33, 34] report that DDIM inversion accumulates big numerical error for CFG γ > 0. Surprisingly, we find that DDIM inversion for CFG γ fwd > 0 can be adequately estimated by running the inversion with negative CFG γ inv = -γ fwd . Figure 10 compares 3D inversion strategies qualitatively and quantitatively. Naïvely taking γ inv = γ fwd = 7.5 yields the biggest numerical error, while other strategies perform on par. 3D generations, illustrated on the right, show that γ inv = γ fwd = 7.5 introduces excessive numerical errors, causing generation to drift in a random direction. The best parameters (as we demonstrate in appendix E) for 2D inversion (γ inv = γ fwd = 0) fail to converge in 3D as there is not enough guidance toward a class sample. Introducing guidance only on the forward pass with γ inv = 0, γ fwd = 7.5 solves the problem, and the algorithm generates the desired 3D shape, but constantly adding CFG on each step over-saturates the image. Note in that configuration the inversion is performed unconditionally from the text prompt, matching ISM [29] . As we can see, prompt conditioning is an important component in eq. ( 8), and using γ inv = -7.5, γ fwd = 7.5 cancels the over-saturation and produces accurate 3D generations. Potential broader impacts. Our work extends existing 2D diffusion models to generation in 3D settings. As such, SDI could marginally improve the ability of bad actors to generate deepfakes or to create 3D assets corresponding to real humans to interact with in virtual environments or games; it also inherits any biases present in the 2D model. While this is a critical problem in industry, our work does not explicitly focus on this use case and, in our view, represents a negligible change in such risks, as highly convincing deepfake tools are already widely available.Appendix A Implementation details.In this section, we provide implementation details for our algorithm. Figure 12 provides a side-by-side comparison of SDS and our algorithm.Algorithm 1 Dreamfusion (SDS)Input: ψ ∈ R N -parametrized 3D shape C -set of cameras around the 3D shape y -text prompt g : R N × C → R n×n -differentiable renderer ϵ (t) θ : R n×n → R n×n -trained diffusion model Output: 3D shape ψ of y procedure DREAMFUSION(y) for i in range(n_iters) do t ← Uniform(0, 1) c ← Uniform(C) ϵ ← Normal(0, I) xt ← α(t)g(ψ, c) + 1 -α(t)ϵ ∇ ψ LSDS = σ(t) ϵ (t) θ (xt, y) -ϵ ∂g ∂ψ Backpropagate ∇ ψ LSDS SGD update on ψAlgorithm 2 Ours (SDI) Our implementation uses the following choice of κ t y :Input: ψ ∈ R N -parametrized 3D shape C -set of cameras around the 3D shape y -text prompt g : R N × C → R n×n -differentiable renderer ϵ (t) θ : R n×n → R n×n -trained diffusion model Output: 3D shape ψ of y procedure OURS(y) for i in range(n_iters) do t ← 1 -i/n_iters c ← Uniform(C) ϵ ← κ t+τ y (g(ψ, c)) xt ← α(t)g(ψ, c) + 1 -α(t)ϵ ∇ ψ LSDS = σ(t) ϵ (t) θ (xt, y) -ϵ ∂g ∂ψ Backpropagate ∇ ψ LSDS SGD update on ψEQUATIONA.1 Timesteps.In eq. ( 10), the added noise inverts eq. ( 8) up to noise level t, but the denoising step happens to slightly lower time t -τ (t 1 and t 2 in fig. 4 , right). Intuitively, τ controls the effective step size of the denoising (or image improvement) process. To accommodate this, we maintain a global time variable t that linearly decays for all the views. Then, on each update step we run DDIM inversion up to t + τ , where τ is a small constant. In practice, we did not find the algorithm to be sensitive to this constant and sample τ ∼ U (0, 1 30 ), where 1 30 is a typical step size in DDIM.The Janus problem is a common issue reported across multiple score distillation works [5, 11, 12] , wherein the model produces frontal views all around the shape due to a bias for views dominant in the training data. SDS [5] tackles this problem by augmenting the prompts with information about the view direction. Our method's variance reduction, however, enabled the use of lower CFG values γ = 7.5 (instead of 100 for SDS), and as shown in fig. 3 , with smaller CFG values, the diffusion model tends to ignore certain parts of the prompts (like "colored" for γ < 10 in fig. 3 ). Thus in our algorithm we observe the diffusion model to be less attentive to the prompt augmentation yielding a stronger Janus problem compared to SDS; we also see the same behavior for other baselines that reduce CFG.To address this issue, we use Perp-Neg [36] and add a small entropy term ∼ N (0, 0.3 1 -α(t)I) (eq. ( 11)) to the inverted noise to reduce mode-seeking behavior as in [44] .Another common issue in Score Distillation is the hollow face illusion [46] , which is a tendency to create holes in the geometry of a visually convex shape. To address this problem, we use a normal map extracted from the volume as described in [5] and downscaled to 8 × 8 resolution so that only low frequencies are penalized. Then, we calculate the sine between adjacent normals (from left to right and top to bottom) and penalize it to be negative. We activate this loss for the first 40% of the training steps with weight α convexity = 0.1 to break the symmetry.Appendix H.1 demonstrates examples of typical failure cases, including the Janus problem and the hollow face illusion.A.3 System details.We implement our algorithm in threestudio [43] on top of SDS [5] . We use Stable Diffusion 2.1 [1] as the diffusion model. For volumetric representation we use InstantNGP [8] . Instead of randomly sampling time t as in SDS, we maintain a global parameter t that linearly decays from 1 to 0.2 (lower time values do not have a significant contribution). Next, for each step we render a 512 × 512 random view and infer κ t+τ by running DDIM inversion for int(10t) steps-i.e. we use 10 steps of DDIM inversion for t = 1 and linearly decrease it for smaller t. We use NVIDIA A6000 GPUs and run each generation for 10k steps with learning rate of 10 -2 , which takes approximately 2 wall clock hours per shape generation.A.4 Prompts used in the quantitative evaluation "A car made out of sushi" "A delicious croissant" "A small saguaro cactus planted in a clay pot" "A plate piled high with chocolate chip cookies" "A 3D model of an adorable cottage with a thatched roof" "A marble bust of a mouse" "A ripe strawberry" "A rabbit, animated movie character, high detail 3d mode" "A stack of pancakes covered in maple syrup" "An ice cream sundae" "A baby bunny sitting on top of a stack of pancakes" "Baby dragon hatching out of a stone egg" "An iguana holding a balloon" "A blue tulip" "A cauldron full of gold coins" "Bagel filled with cream cheese and lox" "A plush dragon toy" "A ceramic lion" "Tower Bridge made out of gingerbread and candy" "A pomeranian dog" "A DSLR photo of Cthulhu" "Pumpkin head zombie, skinny, highly detailed, photorealistic" "A shell" "An astronaut is riding a horse" "Robotic bee, high detail" "A sea turtle" "A tarantula, highly detailed" "A DSLR photograph of a hamburger" "A DSLR photo of a soccer ball" "A DSLR photo of a white fluffy cat" "A DSLR photo of a an old man" "Renaissance-style oil painting of a queen" "DSLR photograph of a baby racoon holding a hamburger, 80mm" "Photograph of a black leather backpack" "A DSLR photo of a freshly baked round loaf of sourdough bread" "A DSLR photo of a decorated cupcake with sparkling sugar on top" "A DSLR photo of a dew-covered peach sitting in soft morning light" "A photograph of a policeman" "A photograph of a ninja" "A photograph of a knight" "An astronaut" "A photograph of a firefighter" "A Viking panda with an axe"In this section, we extensively compare our algorithm with Interval Score Matching (ISM) [29] .Theoretical assumptions. ISM empirically observes that "pseudo-GT" images used to guide Score Distillation Sampling (SDS) are sensitive to their input and that the single-step generation of pseudo-GT yields over-smoothing. From these observations, starting with SDS guidance, ISM adds DDIM inversion and multi-step generation to empirically improve the stability of the guidance. In contrast, our work starts with 2D diffusion sampling to re-derive score-distillation guidance and motivate improvements. That is, our work formally connects SDS to well-justified techniques in 2D sampling.In ISM, the empirically motivated DDIM inversion is at the basis of the derivation of the final update rule. We suggest a general form of the noise term eq. ( 8), for which DDIM inversion is just one possible solution. Our theoretical insights are agnostic to particular algorithms of rootfinding, which makes it possible to use more efficient solutions in future research (e.g. train diffusion models as invertible maps to sample noise faster).Guidance term. The update rules provided in our eq. ( 10) and ISM's eq.17 have two main differences:1. Full vs. interval noise. Assuming DDIM inversion finds a perfect root of our stationary equation, ISM's update rule can take a similar form to ours eq. ( 10) (interval noise is equal to κ if eq. ( 8) is satisfied). However, as shown in fig. 9 , DDIM inversion does not find a perfect root, and thus the two forms are not equivalent. Our theory shows that the full noise term is more accurate. We also show the effect of choosing one term vs. another in fig. 14 . 2. Conditional vs. unconditional inversion. Our derivation hints that the roots of eq. ( 8) are prompt-dependent, motivating our use of conditional DDIM inversion (not used in ISM). Our fig. 10 shows how unconditional inversion yields over-saturation. Summary. We bridge the gap between experimentally-based score distillation techniques and theoretically-justified 2D sampling. Both SDS and ISM can be seen as different approaches to finding roots of eq. ( 8). This theoretical insight allows us to modify both ISM and SDS, reducing over-saturation for the first and improving general quality of the second.Next, we explore the diversity of the results generated by our method. Figure 15 depicts generated shapes for different seeds and prompts. Generations with our method exhibit a certain degree of diversity in local details, but are mostly the same at a coarse level. This is similar to the reported results in other works on score distillation [5, 11] . We plan to address this problem in future research. Additionally, we noticed a minimal diversity in human generations. For example, fig. 16 demonstrates generated shapes for 3 prompts related to different professions.Figure 16 : Examples of limited diversity across different human-related prompts: "a photo of a firefighter", "a photo of a policeman", "a photo of a soldier."In this section, we provide additional examples of the noise samples obtained with different κ t y . As can be seen from fig. 17 , optimization with SGD almost does not change the noise pattern compared to random sampling. Fixed-point iterations drift in a notable but counterproductive direction, and the single-step prediction from the obtained noisy images (depicted on the right side) degrades. DDIM inversion with γ inv = 7.5 and γ inv = -7.5 produces similar noise patterns located around the "meaningful" parts of the image; however, γ inv = -7.5 results in noise patterns that are much more compatible with the single-step denoising procedure used in SDS, leading to much more accurate details appearing for earlier t. In this section, we provide additional experiments testing the DDIM inversion strategy and study the performance of different combinations of the CFG value on the inversion stage (γ inv ) and on the forward pass (γ fwd ). In particular, in fig. 18 we at first generate an image with 30 steps of DDIM (left-most image) and γ = 7.5. Then we run DDIM in the opposite direction of t with γ inv all the way to t = 1. Finally, we use the obtained noise sample as an initialization for a new DDIM inference with γ fwd . Ideally, the final image will correspond to the input of the inversion algorithm.Figure 18 shows that when inverting the DDIM trajectory starting with a clean image, the best inversion strategy uses γ inv = 0 and then regenerates the image via with γ fwd = 0. [34] suggests this approach, which inverts the computation almost perfectly. Other approaches introduce bias and only approximate the image on the forward pass.As we have seen in section 6.2, in 3D, the story is different: γ inv = γ fwd = 0 fails to generate any meaningful shape. We explain this behavior by the fact that in our 2D experiment, the original image contains class information; inverting with γ inv = 0 encodes this information into the noise and thus does not need additional CFG during reconstruction. In 3D generation, however, the initial NeRF does not contain class information and needs bigger CFG on the forward pass. Additionally, fig. 18 demonstrates the inversion quality of an entire DDIM trajectory, whereas eq. ( 10) we are interested in inverting a single-step denoising only. what might explain the fact that seemingly bad inversion quality of γ fwd = -γ inv = 7.5 yields much better results in 3D.Input 30 its 30 its 300 its 30 its 30 its 3018: Comparison of different DDIM inversion strategies in 2D. Here, we compare the quality of inversion, i.e., the forward pass is the entire DDIM trajectory.Our derivation in section 4 manipulates the time-stepping procedure of DDIM, but a similar argument applies to the ODE version of the method. In particular, one can obtain an alternative form of eq. ( 10) by reparametrizing the ODE eq. (1). To do the change of variables in eq. ( 1) to eq. ( 5) we need to differentiate it with respect to t and evaluate dx(t) dt . Direct differentiation gives us:EQUATIONSolving for dx(t) dt and merging with eq. ( 1), we get:EQUATIONOr, expressed in other scaling parameters:EQUATIONThis expression gives us the DDIM's ODE re-parametrized for the dynamics of x 0 (t). This ODE is a velocity field that projects a starting image to a conditional distribution learned by the diffusion model. Discretizing this equation leads to eq. ( 10).Here we provide an intuition why it is so important to regress the noise κ instead of randomly sampling it. In particular, we can look into the data that was used to train ϵθ . For each t, the model has seen only clean images with the corresponding amount of noise σ(t).Consider a noisy or blurry image x0 . Our goal is to use our updated ODE to improve it and make it look more realistic. For t close to 1, the amount of noise is so big, that for any image x0 its noisy version x0 + σ(t)ϵ looks indistinguishable from the training data of the denoiser. This explains why we can successfully use diffusion models to correct out-of-distribution images.For small t, however, the noise levels might be not enough to hide the artifacts of x0 , so the image becomes out-of-distribution, and the predictions of the model become unreliable. We hypothesize, that if instead of randomly sampling ϵ like in Dreamfusion, we can find such a noise sample, that under the same noise level t the artifacts of x0 will be hidden and it will be in-distribution of the denoiser. This effectively increases the ranges of t where the predictions of the model are reliable and thus improves the generation quality. This intuition is illustrated in fig. 19 . Lastly, we discuss failure cases and showcase additional generations using our methods, along with more baseline comparisons.We provide illustrations of typical failure cases of our algorithm in Figures 20 to 23 .Figure 20 : Content Drift: due to a limited communication between the frames sometimes we can observe that certain properties "drift" around the shape. In this particular example the number on the fireman's hat is different from all the views.Figure 21 : Hollow Face Illusion: some shapes might be generated with a concave hole in the geometry, while the rendering appears to be convex. Figures 24 to 27 provide additional generations produced using ourWe provide additional qualitative comparisons with other score distillation algorithms in figs. 28 to 30. We compare with Dreamfusion [5] , Magic3D [14] , Fantasia3D [47] , Stable Score Distillation (StableSD [27] , Classifier Score Distillation (ClassifierSD [16] ), Noise-Free Score Distillation (NFSD [12] ), ProlificDreamer or Variational Score Distillation (VSD [11] ), Interval Score Matching (ISM) [29] , and HiFA [13] ."a photograph of a knight" "a ripe strawberry" "a ceramic lion" "a photograph of a ninja" "Tower Bridge made out gingerbread and candy" DSLR photo a beautifully decorated cupcake with on top" • The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and that the proof relies upon should be properly referenced. 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: As a procedure for sampling from preexisting diffusion models, our method, to our knowledge, should need no additional safeguards beyond those already appropriate for the preexisting diffusion models. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: The code for implementing our method is generated by ourselves on top of threestudio [43] , any preexisting diffusion models used are properly cited and the relevant licenses followed. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL.• The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets? Answer: [Yes] Justification: The code we released is documented. Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
: 
 "pumpkin head zombie, skinny, highly detailed, photorealistic""bagel filled with cream cheese and lox""photograph of a black leather backpack" "a DSLR photo of Cthulhu""a DSLR photo of an old man"Figure 2 : Examples of 3D objects generated with our method. 
 
 Appendix:  


Decision: The paper introduces Score Distillation via Inversion (SDI), a method that enhances 3D shape generation by improving the Score Distillation Sampling (SDS) process. SDI connects SDS with Denoising Diffusion Implicit Models (DDIM), reducing over-smoothing and preserving detail in 3D models. This approach improves 3D generation quality to match that of 2D images without needing additional training or multi-view supervision. SDI connects SDS with DDIM, providing deeper insights into the discrepancies between 2D and 3D generation. The method improves 3D generation quality without the need for additional training or complex setups. After carefully reading the paper, reviews, and rebuttals, the AC recommends to accept the paper.

Reviews: review: paper_summary: paper_summary: This paper connects score distillation sampling (SDS) to a DDIM sampling process. The proposed method Score Distillation via Inversion (SDI) replaces the original random noise in SDS with DDIM inversion, and show significantly improved quality compared to SDS and other state-of-the-art prior methods.
strength_weakness: strength_weakness: 1. Overall, the visual results seem to be having a little bit "gray" color/style shift. Is this due to the approximation or some bias in the theory? Is there any hypothesis for theoretical / practical reasons?
2. The ddim inversion and approximation could introduce additional computation cost and training noise.
3. In "ISM as a special case", it seems the main difference of the proposed method and ISM is having the additional text condition. Is there any other key differences in method implementation or theory? More details about the contribution of this work based on ISM could be helpful.
questions: questions: Does the proposed method remedy Janus problem, compared to the baseline SDS method? (If yes, why)
Also please check the weakness section.
limitations: limitations: The authors discussed limitations.
paper_summary: paper_summary: The paper introduces an effective SDS modification that replaces random noise samples in the SDS objective with those obtained with DDIM inversion. The proposed technique enhances text-to-3D generation, outperforming SDS and being comparable to more sophisticated methods.
strength_weakness: strength_weakness: * Inversion with negative guidance is surprising, and its rationale remains somewhat unclear. A more detailed investigation and justification would be useful.

* The idea of using DDIM inversion is very similar to ISM[1]. The primary distinction is guided vs unguided inversion. Although the authors justify the guided inversion with negative guidance in Fig. 10, a thorough discussion and detailed comparisons are essential.

* The quantitative evaluation relies on CLIP-IQA, which does not seem quite suitable for 3D generation evaluation. Conducting a user study would be highly valuable. If it is unavailable, some recent works [2,3] aim to propose automated evaluation metrics. For example, would it be reasonable to evaluate ImageReward[4] following [2]?

* The quantitative results miss NFSD and the qualitative comparisons are presented only for two prompts. Could the authors add NFSD to Tab.1 and provide more visual comparisons with all methods?

* There is no evaluation of diversity under a given prompt. It would be useful to include a few generated results for different seeds.

To sum up, I am generally optimistic about the submission but have some concerns regarding the evaluation and close connection with [1], as discussed above.

[1] Liang et al. LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching.

[2] He et al. T3Bench: Benchmarking Current Progress in Text-to-3D Generation.

[3] Wu et al. GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation.

[4] Xu et al. ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation.
questions: questions: * Please address the questions and concerns raised in "Weaknesses".

* In Fig.5, the intermediate $x_t$ do not have noise in the background. DDIM inversion must follow the same marginal distributions as the forward process, but the presented $x_t$ do not. We observed the similar effect for images with monotonic regions if inversion is applied directly to a clean image. We believe such images are likely OOD for the pretrained DM, and injecting slight noise with $\sigma_{min}$ to an image fixed the problem. Do the authors have any thoughts on this? Would the solution above work for SDI and how does it affect its performance?
limitations: limitations: The authors addressed the limitations and potential negative social impact.
paper_summary: paper_summary: This paper introduces a novel algorithm called Score Distillation via Inversion (SDI) that enhances the quality of 3D shape generation. By inverting Denoising Diffusion Implicit Models (DDIM) at each step and incorporating the initial noise into the estimated score, SDI addresses the over-smoothing and detail loss issues associated with traditional Score Distillation Sampling (SDS) methods in 3D generation. Experimental results demonstrate that SDI significantly improves the quality of 3D shapes, closely matching the quality of 2D image generation, without the need for additional neural network training or multi-view supervision. The work provides valuable insights into understanding and improving 3D asset generation with diffusion models.
strength_weakness: strength_weakness: 1.	According to the authors’ theory and hypothesis, the better the noise term is Eq. (8) is estimated, the better the 3D generation quality is improved. However, in the ablations about the choice of k(x), SGD optimization doesn’t show significant improvement. Also, this phenomenon can be observed from Figure.10, in which green curve and purple curve present lowest and comparable MSE with orange curve, while the orange curve get the best result. I would appreciate it if the authors can provide some explanations.
2.    One of the paper's key perspective is that SDS introduces high variance. I have read another paper[1] dedicated to  minimizing the variance introduced in SDS, and maybe it's also worthwhile to compare with that.



[1] Tang, B., Wang, J., Wu, Z., & Zhang, L. (2023). Stable score distillation for high-quality 3d generation. ArXiv Preprint ArXiv:2312.09305.
questions: questions: Please refer to the weaknesses part.
limitations: limitations: Please refer to the weaknesses part.

Comments: comment: Thank you for the high evaluation of our work! We are glad that we were able to address the comments and will include a compilation of our answers into the final version.
Thank you, we appreciate your input and the acknowledgment of our work!
Thanks for the authors' detailed response and explanation! My questions have been largely addressed and I also learned a lot. I hope the authors can summarize these points into the final version. I have increased my score accordingly.
I would like to thank the authors for their extensive clarifications, discussions and additional evaluations. My questions have been largely addressed. Overall, I feel more confident about the submission and have increased my score accordingly.
Dear reviewer ZvgL,
Thank you again for the recommendations! Could you please let us know if we fully addressed the comments in the review and/or if there is any additional information we could provide? Please note that the author-reviewer discussion period will be over on Tuesday at 23:59 AoE and we won't be able to further reply after this point.

Thank you in advance!
On behalf of all authors
Dear reviewer r6QS,

Thank you again for the thorough review and high evaluation of our work. Could you please let us know if our response fully addressed your comments and if we can add any additional clarifications ? Please note the additional evaluations we added in the PDF attached to the general response.

Best regards,
On behalf of all authors
Dear reviewer 4KTy,

Given the limited time for the author-reviewer discussion period, we would appreciate if you could let us know if we fully addressed your comment and if there are any additional questions or clarifications we could add.

Thank you again for your time and effort.

Best regards,
On behalf of all authors