Paper: SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via Stein Identity (ICLR-2024)

Reviewer comment (Theory):
My main concern is that why a lower variance during optimization is helpful for the final quality? Although there are some empirical results in Sec 3 show that there are some corelation between variance and generated quality, I think a more convincing justification should be given. Maybe a more detailed theoretical or intuitive explanation should be given.

Author response:
In optimization theory, variance plays a crucial role in determining the convergence rate of SGD algorithms (refer to Theorem 5.3 in [1]). With a finite number of optimization steps and a standard learning rate, maintaining low variance is pivotal to ensure convergence. Training with noisy gradients introduces high instability, potentially resulting in suboptimal solution or even divergence, as illustrated in Fig. 4(a).

[1] Garrigos & Gower, Handbook of Convergence Theorems for (Stochastic) Gradient Methods

Reviewer comment (Evaluation):
No quantitative results are given to compare the proposed method and baselines in terms of the visual quality.

Author response:
We present a qualitative comparison among DreamFusion, ProlificDreamer, and SteinDreamer across all showcased scenes, utilizing the CLIP score as our metric. 

| method              |   tulip |   sushi car |   lionfish |
|:--------------------|--------:|------------:|-----------:|
| DreamFusion      |   0.777 |       0.862 |      0.751 |
| ProlificDreamer     |   0.751 |       0.835 |      0.749 |
| SteinDreamer  |   0.734 |       0.754 |      0.735 |

| method              |   cheesecake castle |   dragon toy |   michelangelo dog |
|:--------------------|--------------------:|-------------:|-------------------:|
| DreamFusion      |               0.902 |        0.904 |              0.789 |
| ProlificDreamer     |               0.843 |        0.852 |              0.775 |
| SteinDreamer |               0.794 |        0.806 |              0.769 |

Our observations indicate that SteinDreamer consistently outperforms all other methods, which improves CLIP score by ~0.5 over ProlificDreamer. This superior result suggests our flexible control variates are more effective than the one adopted by ProlificDreamer.

Reviewer comment (Theory):
How will the baseline function effects the final results? Since the Stein identity always holds, what is the relationship between the baseline function and the final performance?

Author response:
Various baseline functions can yield differing correlations between the constructed control variate and the target random variable for estimation. As outlined in Section 2.2, it is advisable to employ a baseline function that exhibits a stronger correlation with the 2D diffusion model to minimize variance. Typically, we notice that baseline functions with a more robust prior tend to demonstrate higher correlations. From another perspective, the baseline function $\phi$ can be regarded as a guidance introduced into the distillation process. Intuitively, enforcing priors and constraints on the gradient space can also stabilize the training process by regularizing the optimization trajectory. In our empirical design, the inclusion of geometric information expressed by a pre-trained MiDAS estimator is expected to result in superior variance reduction compared to SSD and VSD.

Reviewer comment (Experiments):
The experimental results are not as extensive as ProfilicDreamer. Compared with 10 object and 8 sence level generated content on https://ml.cs.tsinghua.edu.cn/prolificdreamer/, there are only 6 object and 4 sence level in this work and supplementary demo.

Author response:
We additionally showcase four qualitative results on scene generation in Appendix C.4. Aligning with our prior empirical observations, our method consistently produces sharp and visually coherent results.

Reviewer comment (Experiments):
Looking at the Figure 5, there existing the following confusions: 1) DreamFusion seems not converge for some cases, especially for "A Car made out of Sushi." and "A Lion Fish". Maybe DreamFusion needs more iterations?

Author response:
Every qualitative result, including DreamFusion's, was obtained after 25k training steps, which follows the standard training configuration in the threestudio repository. To be more convincing, we train DreamFusion with the prompt "a car made out of sush" for an additional 10k steps, there were no observable improvements in the results. See our Appendix C.3 for more details.

Reviewer comment (Evaluation):
Looking at the Figure 5, there existing the following confusions: 2) The proposed SteinDreamer can be overfitting after ~120k training steps, with gradually increasing variance. How could this happen?

Author response:
As depicted in Fig. 5, not all scenarios witness an increase in variance, as seen in examples like 'a blue tulip' and 'a plush dragon toy'. We emphasize that the optimization trajectory of score distillation exhibits high non-smoothness, leading to a typical fluctuation phenomenon in variance. Even in instances where variance appears to rise, its final value remains upper bounded, ensuring the convergence of our results. In contrast, other methods, such as ProlificDreamer with the prompt 'a plushy dragon toy', display potential variance explosion, evident in the training instability depicted in Fig. 4(a).

Reviewer comment (Evaluation):
For comparing the convergence speed, it is much better to compare when all the methods are fully converged, especially for DreamFusion and ProfilicDreamer.

Author response:
We express our apologies for the mistake in Fig. 7, where the maximum of x-axis was incorrectly set to 100 instead of the intended 25k steps. This initial error occurred as we mistakenly used the indices of data points for the x-axis. We have rectified this issue in our revised version. It is important to note that 25k is the default setting suggested by threestudio, typically resulting in converged results with ProlificDreamer. To verify this claim, we train ProlificDreamer for an additional 10k steps. Both visual quality and the CLIP score exhibit negligible changes. See Appendix C.3 for more details.

Reviewer comment (Writing):
Possible typos: 4th para. in Inroduction, 'aligns with with that' --> 'aligns with that'

Author response:
We have fixed all the typos and addressed your concerns as follows.

Reviewer comment (Experiments):
One biggest weakness of this work is the insufficient empirical evaluation. The variance plot is one face of the story. However, it would greatly strengthen the work if the authors can show the 3d synthesis model learned by SSD is quantitatively better than the baselines given the same compute. In Figure 7, the improvement is very marginal considering the additional cost of training the control variates.

Author response:
**2. Quantitative comparison with other methods.**

We present a qualitative comparison among DreamFusion, ProlificDreamer, and SteinDreamer across all showcased scenes, using the CLIP score as our metric. 

| method              |   tulip |   sushi car |   lionfish |
|:--------------------|--------:|------------:|-----------:|
| DreamFusion      |   0.777 |       0.862 |      0.751 |
| ProlificDreamer     |   0.751 |       0.835 |      0.749 |
| SteinDreamer  |   0.734 |       0.754 |      0.735 |

| method              |   cheesecake castle |   dragon toy |   michelangelo dog |
|:--------------------|--------------------:|-------------:|-------------------:|
| DreamFusion      |               0.902 |        0.904 |              0.789 |
| ProlificDreamer     |               0.843 |        0.852 |              0.775 |
| SteinDreamer |               0.794 |        0.806 |              0.769 |

Our observations indicate that SteinDreamer consistently outperforms all other methods, demonstrating an approximate 0.5 improvement in CLIP distance over ProlificDreamer.

Reviewer comment (Novelty):
It is noted that the primary focus of this paper is to interpret score distillation sampling using the Stein identity and to reduce variance through the incorporation of monocular depth estimation. However, Kim et al [1] have already discussed a similar interpretation of the Stein identity and the importance of baseline function selection for convergence and efficiency. Although the specific focus of Kim et al. is different (visual editing), the core inspiration appears to be similar. Therefore, it is recommended for the authors to acknowledge and discuss the already incorporated findings of Kim et al. In this context, while the inclusion of monocular depth estimation for efficient text-to-3D synthesis is intriguing, the novelty of the proposed method may be considered weak. Thus, given the similarities in findings and interpretation, the distinguishing factor appears to be only the choice of baseline functions for specific tasks, the adoption of a monocular depth estimator as the control variate.

Author response:
**1. Lack of novelty and idea coincides Kim et al.**

Before submission, we were aware of Kim et al.'s work [1]. However, we respectfully disagree regarding the similarity of their method to ours. While both methods are grounded in Stein's method, the underlying principles significantly differ. In their approach [1], the SVGD-based update takes the form of the Stein discrepancy: $\max_{\phi \in \mathcal{F}} \mathbb{E}\_{x\sim q(x)} [\phi(x) \nabla \log p(x) + \nabla \phi(x)]$, where $\phi(x)$ is often interpreted as an update direction constrained by a function class $\mathcal{F}$ (RBF RKHS in [1]). In contrast, our update rule appends a zero-mean random variable via the Stein identity after the raw gradient of the KL divergence: $\mathbb{E}_{x \sim q(x)} [\nabla \log p(x) + \phi(x) \nabla \log q(x) + \nabla \phi(x)]$, where $\phi(x)$ typically represents a pre-defined baseline function. The potential rationale behind [1]'s approach to reducing variance lies in introducing the RBF kernel as a prior to constrain the solution space by modeling pairwise relations between data samples. Our approach to reducing variance is centered around constructing a more general control variate that correlates with the random variable of interest, featuring zero mean but variance reduction. We have cited [1] and included the above discussion in our revised submission (Sec. 4.3).

[1] Kim et al., Collaborative Score Distillation for Consistent Visual Editing. NeurIPS 2023.

Reviewer comment (Experiments):
The comparison in the paper appears to be based on a limited number of samples. It is beneficial to conduct experiments with a broader range of prompts to assess the robustness of the proposed method in reducing variance.

Author response:
**4. The number of examples is limited.**

To enhance our experiment results, we present an additional set of four qualitative results on scene generation in Appendix C.4. As previously noted, our method consistently delivers smooth and visually consistent outcomes. Their video demos can be found in our updated supplementary materials.

Reviewer comment (Novelty):
As demonstrated in the Weaknesses section, the authors should make a comparison with the prior work of Kim et al. and discuss any differences if there exists new interpretation.

Author response:
**1. Lack of novelty and idea coincides Kim et al.**

Before submission, we were aware of Kim et al.'s work [1]. However, we respectfully disagree regarding the similarity of their method to ours. While both methods are grounded in Stein's method, the underlying principles significantly differ. In their approach [1], the SVGD-based update takes the form of the Stein discrepancy: $\max_{\phi \in \mathcal{F}} \mathbb{E}\_{x\sim q(x)} [\phi(x) \nabla \log p(x) + \nabla \phi(x)]$, where $\phi(x)$ is often interpreted as an update direction constrained by a function class $\mathcal{F}$ (RBF RKHS in [1]). In contrast, our update rule appends a zero-mean random variable via the Stein identity after the raw gradient of the KL divergence: $\mathbb{E}_{x \sim q(x)} [\nabla \log p(x) + \phi(x) \nabla \log q(x) + \nabla \phi(x)]$, where $\phi(x)$ typically represents a pre-defined baseline function. The potential rationale behind [1]'s approach to reducing variance lies in introducing the RBF kernel as a prior to constrain the solution space by modeling pairwise relations between data samples. Our approach to reducing variance is centered around constructing a more general control variate that correlates with the random variable of interest, featuring zero mean but variance reduction. We have cited [1] and included the above discussion in our revised submission (Sec. 4.3).

[1] Kim et al., Collaborative Score Distillation for Consistent Visual Editing. NeurIPS 2023.