RecDreamer: Consistent Text-to-3D Generation via Uniform Score Distillation
Authors: Chenxi Zheng
Yihong Lin
Bangzhen Liu
Xuemiao Xu
Yongwei Nie
Shengfeng He
Venue: ICLR 2025 Poster (2025)
Keywords: 3D generation
Primary area: generative models
TL;DR: A new text-to-3D generation method with superior view-consistency.

Abstract: Current text-to-3D generation methods based on score distillation often suffer from geometric inconsistencies, leading to repeated patterns across different poses of 3D assets. This issue, known as the Multi-Face Janus problem, arises because existing methods struggle to maintain consistency across varying poses and are biased toward a canonical pose. While recent work has improved pose control and approximation, these efforts are still limited by this inherent bias, which skews the guidance during generation.
To address this, we propose a solution called RecDreamer, which reshapes the underlying data distribution to achieve more consistent pose representation. The core idea behind our method is to rectify the prior distribution, ensuring that pose variation is uniformly distributed rather than biased toward a canonical form. By modifying the prescribed distribution through an auxiliary function, we can reconstruct the density of the distribution to ensure compliance with specific marginal constraints. In particular, we ensure that the marginal distribution of poses follows a uniform distribution, thereby eliminating the biases introduced by the prior knowledge.
We incorporate this rectified data distribution into existing score distillation algorithms, a process we refer to as uniform score distillation. To efficiently compute the posterior distribution required for the auxiliary function, RecDreamer introduces a training-free classifier that estimates pose categories in a plug-and-play manner. Additionally, we utilize various approximation techniques for noisy states, significantly improving system performance.
Our experimental results demonstrate that RecDreamer effectively mitigates the Multi-Face Janus problem, leading to more consistent 3D asset generation across different poses.

Published as a conference paper at ICLR 2025
RECDREAMER: CONSISTENT TEXT-TO-3D GENERATION VIA UNIFORM SCORE DISTILLATION
Chenxi Zheng', Yihong Lin', Bangzhen Liu', Xuemiao Xu't, Yongwei Nie Shengfeng He² 'South China University of Technology, "Singapore Management University {chansey0529, amesyihonglin, liubz.scut] @ gmail.com, shengfenghe@@n @ smu.edu.sg
( (xuemx, nieyongwei) @ssut.edu.cn,
ABSTRACT
Current text-to-3D generation methods based on score distillation often suffer from geometric inconsistencies, leading to repeated patterns across different poses of 3D assets. This issue, known the Multi-Face Janus problem, arises because existing methods struggle to maintain consistency across varying poses and are bi- ased toward a canonical pose. While recent work has improved pose control and approximation, these efforts are still limited by this inherent bias, which skews the guidance during generation. To address this, propose solution called RecDreamer, which reshapes the underlying data distribution to achieve a more consistent pose representation. The core idea behind method is rectify the prior distribution, ensuring that pose variation is uniformly distributed rather than biased toward canonical form. By modifying the prescribed distribution through auxiliary function, we can reconstruct the density of the distribution to ensure compliance with specific marginal constraints. In particular, we ensure that the marginal distribution of poses follows uniform distribution, thereby eliminat- ing the biases introduced by the prior knowledge. We incorporate this rectified data distribution into existing score distillation algorithms, a process we refer to as uniform score distillation. To efficiently compute the posterior distribution re- quired for the auxiliary function, RecDreamer introduces a training-fri -fee classifier that estimates pose categories in a plug-and-play manner. Additionally, we utilize various approximation techniques for noisy states, significantly improving system performance. Our experimental results demonstrate that RecDreamer effectively mitigates the Multi-Face Janus problem, leading to more consistent 3D asset gen- eration across different poses.
INTRODUCTION
Text-to-3D generation has become transformative technology with broad applications, enabling the creation of 3D models from natural language descriptions. By lowering the technical barri- ers, it allows non-experts generate intricate 3D objects without specialized tools expertise. This advancement significantly enhances productivity in fields such as gaming, virtual reality (VR), and augmented reality (AR), where manual 3D model creation is often labor-intensive. Current methods (Wang et al. 2024b Chen et al., 2023 Lin et al. 2023) rely on score distillation tech- niques (Poole et al. 2022] Wang et al.) [223a] Graikos et al., 2022) to leverage text-to-image priors from diffusion models, generating high-quality 3D assets with remarkable visual fidelity, precise alignment to text descriptions, and strong conceptual integrity.
However, despite these advances, generated 3D assets frequently suffer from geometric inconsisten- cies, particularly the form of repeated patterns or textures across different camera poses, a problem known the Multi-Face Janus issue (see Fig. (b)). This arises from biases in the underlying data distribution (sce Fig. [[a),, which current methods fail to fully address. Efforts to tackle this issue, such as modifying directional text descriptions through gradient- based adjustments (Hong et al. 2023 Armandpour et al. 2023), have yielded limited success, often introducing unwanted artifacts inrelevan patterns. Other approaches (Huang al. 2024 Wang et al.) 2024a) attempt to impose
Corresponding authors. Code: https: / htggttub....eog.g wtt.th...b...ctt.. wtt.:cooocccccccssceee 9/ Rechreamer.

---

Published as a conference paper at ICLR 2025
tront
side
back
0.8
carera
6.8
0.8
ker gance
Pobbabiity
0.0-
6.6
0.6
0.2
0.2
0.2
fronk
Sde
back
Ponn
side
back
front
side
back
Category
Categery
Category
(a) Statistics for pose classification in images generated by diffusion models. The plots show that the overall distribution is biased toward front-facing poses, when directional text prompts used to specify other poses.
(b) Score distillation from biased distribution leads to Multi- Multi-Face Janus problem. The generated 3D assets tend to overemphasize frontal features to align with the prior distribution, resulting in repeated patterns.
(c) Score distillation using uniform distribution with RecDreamer. The process relies rectified prior distribution that incorporates guidance various poses, effectively alleviating geometric inconsistencies.
The Multi-Face Janus problem arises from imbalance in the pose distribution of pre- trained models, which tend generate predominantly frontal images. This bias results in excessive faces appearing in the generated 3D assets. RecDreamer addresses this issue by producing dis- tribution with a uniform pose marginal, enabling more diverse pose generation and mitigating the Multi-Face Janus problem.
Figure 1:
2

---

Published as a conference paper at ICLR 2025
constraints on the rendered 3D assets, but they still fall short of resolving the core bias present in text-to-image distributions.
To address this, propose RecDreamer, novel solution designed to climinate the biases in pre- trained models by modifying the underlying data distribution. The rationale behind our approach is reconstruct the original data distribution so that the marginal distribution of the pose becomes uni- form, thus removing the bias toward a canonical pose. We achieve this by introducing a weighting function that reweights the density of the original distribution, ensuring it meets specific marginal constraints. Specifically, we derive rectified distribution where the pose component in the joint distribution follows a uniform distribution across all possible poses.
This rectified distribution is then incorporated into the score distillation framework (Wang et al., 2024b). The use of reverse Kullback-Leibler divergence (Kullback Leibler, [951] score dis- tillation allows the integration of the modified distribution without altering the overall sampling process or gradient derivation. As result, we develop process known as uniform score distillation (USD), which aligns the target distribution with uniform distribution, effectively improving pose consistency the generated 3D assets.
To compute the auxiliary function necessary for rectifying the distribution, RecDreamer introduces training-free classifier that estimates pose categories by discretizing the continuous pose space. This classifier predicts pose based on orientation score and texture similarity, leveraging pretrained fea- ture extractor without the need for additional fine-tuning. Furthermore, we dynamically handle noisy image estimates, ensuring robust pose estimation and reliable performance in the optimization process.
Experiments demonstrate the effectiveness of our method in alleviating the Multi-Face Janus prob- lem and improving geometric consistency, while maintaining rendering quality comparable to base- line methods, shown in Fig. [.). We also conduct additional experiments 2D images and toy dataset to further validate our algorithm. Additionally, we showcase further applications of the pose classifier.
2 BACKGROUND
In this section, provide brief overview of diffusion models, conditional guidance techniques, and text-to-3D generation using score distillation. We follow the notation conventions introduced in VSD (Wang et al. 2024b).
2.1 DIFFUSION MODELS
Diffusion models - Dickksitt et al. 2015 Ho et al. 2020 Song et al. 2020) are latent variable models that simulate diffusion process to model the data distribution æg qo (2)). These models consist of forward process which progressively adds Gaussian noise to the data, and reverse process p. which denoises the data to recover the original distribution.
In the forward process, noise is iteratively added through transitions ze (xxa////) (2.-)) This allows the posterior distribution qe (a,, (ra) = A (a of to be computed, where az and or are time- dependent hyperparameters, The marginal distribution (<(xx) is derived by integrating over the data distribution:
/
qe (x.)
qe (= dxy.
The reverse process begins with a standard Gaussian distribution, (r())) = NO, I). and removes
the noise through transitions Pt (x,|<<++1) ge [I,41, clean data. Instead of predicting &g directly, Ho et al. 2020 proposed optimizing a noise estimator
1, Ty Le). where 2. is an estimate of the
ca( by minimizing the following loss function:
Liif (Φ) En, (ttp--/o_f.o...2000011)ee--00000 (a +,,e) - -E112]
where wt) is dime-dependent weighting function. The noise predictor ca(2,, t) can be viewed as linear transformation of the score function: 6 008 -
3

---

Published as a conference paper at ICLR 2025
2.2 TEXT-TO-3D GENERATION WITH SCORE DISTILLATION
To enable text-to-3D generation using a text-to-image prior, Poole et al. 2022) proposed aligning the distribution of rendered images from an optimizable 3D representation 0 with the text-to-image distribution ( y)) generated by a pretrained diffusion model. Let è represent the space of scene parameters, I2 the space of poses, and R the space of images. Given a pose e and differentiable renderer g(:-)) ④ x IR + R the distribution of the rendered image is computed through the forward process:
(xx(c) = / (2ooc)) (e (oo
where 2g = g(0.c) is the rendered image. The 3D representation then optimized using a weighted probability density distillation loss:
min (0) = Eg,0 (2) w(t) (ol pl (2(a))))))) 6E8
(4)
where o.g((, + orc, and yf is text prompt corresponding to the pose. Since (a is a Dirac distribution o (aa 8(20 - g( e)) (Wang al. 2024a). the gradient of Eq. can be
simplified through reparameterization (Ho et al. 2020) as:
0g(0,c) c)
VoLSDS (0) = E,,c, (t) (fpretmiinaa 1, 6)
(5)
00
where e N', I) and Epretmin is the pretrained diffusion denoiser. During optimization, Eq. 5 results in mode-seeking behavior toward the text-to-iman to--tte--igeeeeee distribution (r which causes over- smoothness and over-saturation in the generated 3D scene.
To address these issues, Wang et al. (2024b) expanded the point estimate of 3D parameters into a more expressive distribution (a(8) by introducing multiple particles (0)) This extends the
simple Gaussian distribution qf (I>, e) in SDS to a more complex distribution:
of (2 v) gt' (zolc, ( (zo (xo)
(6)
The distribution μ then optimized using a variational score distillation (VSD) objective:
arg min Bec at a4
DKL(
pe(ax.1y9))
To solve this, VSD employs particle-based variational inference (Chen et al. 2018 Liu & Wang 2016 and fine-tunes additional U-Net (Ronneberger et al. 2015), 6s. using LoRA (Hu et al. 2021 The fine-tuning is formulated as:
min F...tuf..r.e...o.......oo (1ca(ang(00)) + a.e,t,c,,)) el[3]
(8)
Finally, the gradient for each particle 0 is computed
VeLvsD EE,,,c ((()) (x;, ( 0g(0.c)
(9)
00
METHOD
The primary goal of our RecDreamer to mitigate the Multi-Face Janus problem through rectifi- cation of underlying data distribution in the pre-trained diffusion models. In the following sections, we will first theoretically illustrate the idea of how we rectify the data density via an auxiliary func- tion to ensure a uniform pose distribution (Sec. 3.1.p Based the former theoretical analysis, introduce a uniform score distillation approach for optimizing 3D representations in aligning with the rectified distribution (Sec. 3.2). Furthermore. series of designed components for implementing the auxiliary function is detailly discussed in Sec. [3.3 including a pose classifier, approximation of the posterior distribution of pose, and estimation of pose-relevant statistics.

---

Published as a conference paper at ICLR 2025
3.1 RECTIHICATION OF DATA DISTRIBUTION
To directly analyze the relationship between data and pose, we eliminate redundant variables and simplify the text-conditioned probability pr(xx.ly) influence of the time step. We denote the data with a general variable æ. Assuming that p(ær, c) represents the joint distribution, the pose distribution can be expressed as p(c) =f p(x, c)dx p(z))((c(z))d which is not uniform distribution. To mitigate this bias, we frame the simplified problem as follows: given the data distribution p(æ) and the target attribute distribution f(c), how can we adjust p(æ) to a new distribution p(x)) such that p(c) ((<))p(c(2z)d f(c) holds.
unconditional density p(æ:), removing the
By introducing a weighting function to the joint probability p(æ, c). we establish that the original data density can be adjusted as follows.
Theorem 1 (Proof in Appendix B.4]. Let p(æ) denote the data density, p(c)x) the conditional distribution of the attribule e given data a, and p(c) the marginal distribution of e induced by p(ar). Given a target distribution f(c) for the attribute c, we can construct a new data density p(œ) such that the marginal distribution of e under p(x) matches the target distribution f(c). This new density is given by:
/
f(e) p(c) plo plcla) de.
5(x)= p(xx)
(10)
Theorem reveals that the new data density that features a uniformly distributed marginal f() can be computed by the original data distribution and an auxiliary function. Furthermore, Theo-
rem [ can be naturally extended to conditional distributions, as demonstrated in Corollary 2 (see
Appendix B.4p. So far, we have derived the rectified distribution for clean images, p(xool))
However, since score distillation operates in the noise space, our ultimate goal is to reach the rec- tified density of the noisy data. Given the transition pe(x)ly)) = (,o(2;1120) = Nxx, a, I),, we prove that the rectified distributions for any time step share a unified form, presented in the following theorem.
(xx ( where
Theorem (Proof in Appendix B.4 For any t U[00, Tİ, the rectified density of xz is given by: f(cly) (c(cly)
pe( = - /
p(c|zr, y)d
(11)
Theorem reveals that the noisy density of the rectified text-to-image distribution can be expressed as the original noisy density multiplied by an auxiliary function, denoted as r(æ::ly) Specifically, r(xx)ly) s f(ll) (c(c))) p(c) 2,, y)dc.
3.2 UNIFORM SCORE DISTILLATION
We now return to the original variational distillation problem. First, we define set of 3D repre- sentations { also named particles in the later gradient flow simulation. Given the distribu- tion u(0ly) composed of the set (0')= the camera pose c, and the text prompt y, the distribu- tion of noisy rendered images is computed as of (æxlc, y) = =J I (zo c, y ()P:o ( where an = g(0 c). Given the rectified distribution (e( the objective is as follows:
min Ee,e ((a,/a,)w(()) DKL (q." (m (c) (o(() 1
(12)
We refer to this as uniform score distillation (USD), as it seeks to approximate the score of the recti- fied distribution, which is uniformly distributed across the camera poses. To optimize the particles, we derive corollary based on Theorem 2 from VSD (Wang et al.) 2024b):
Corollary (Corollary to Theorem 2 from VSD). For Wasserstein gradient flow minimizing Eq. 12 the gradient for the particles is given by:
VoLUso Vel'voo Ere,c w(t) "V Volor((z)))
(13)
where
0g(0,c) c)
VoL'esD = Er,e, (w(() (Cpreerain (It,, 1, v) - e c 1, C, v))
(14)
06
and x. = (0, c) + O.E.
5

---

Published as a conference paper at ICLR 2025
Input x
Templates (x²)
Stez
B Heement-wwsee multiplication
Texture Similarity
Softmax
F())
Ppose
( fets Spat)
(fhpac) Ulk3)
Sarl
Opeee
Orientation Similarity
Probubility
The architecture of our classifier combines orientation and texture similarities in a dif- ferential "and-gate" manner. Orientation similarity is evaluated using patch-matching distance metric, while texture similarity is calculated via cosine similarity of the [cls] token.
Figure 2:
Since the rectification algorithm is based on reweighting the sub-distributions, it cannot generate content that was not present the original distribution. To ensure that y provides comprehensive distribution including contents of multiple perspectives, we detail the construction techniques in the Appendix B.3
In the optimization process, we follow VSD by iteratively optimizing the U-Net cø and the particles {0') y using Eq. 8 and Eq. 12
3.3 RECDREAMER
The previous sections derive the analytical solution of the rectified distribution and introduce a parameter optimization scheme based score distillation. To apply this scheme optimize the 31> scene, we must also compute the rectification function r(æ; ly). We design an effective classifier to accurately categorize image poses. Finally, account for the effects of noisy states and estimate the posterior distribution of noisy images and its expected value.
Discretization of the pose space. The crux to computing the auxiliary function r(æ:ly) lies in estimating both p(c|æ,,)) y) and (cly)). Since pt (cly) = (ew.<p(::,)))((( p()ppp y) is a term that depends on p(ccca p(c 20, y). we begin by analyzing p(c|2,,,) y), which can be interpreted as pose estimator noisy images. However, obtaining
estimator for noisy images requires additional data and fine-tuning. To address this, we relate the noisy predictor to the clean predictor by following the DPS (Chung al. 2022) formulation: p(c|ær, y) p( ( ())(c/20. d) Thus, we prioritize the design of clean estimator p(c|zo. y) before tackling the noisy case.
Instead of explicitly estimating the camera's 's extrinsic parameters, we propose modeling a simplified pose by categorizing the images into broad pose categories, such as "front", "back", "left" and "right". In the context of USD, these global categories help maintain a rough balance between different poses and promote 3D consistency. Accordingly, we define the auxiliary function in discrete form as follows: re (=(llly) Ea pe(el) Pe (A/., v). where represents the discrete pose category, f(lly) ! and Pę is the parameterized classifier.
((ll))
Pose classifier. Building on this formulation, our goal is to create a lightweight pose classifier without the need for training. To achieve this, we propose matching-based pose classifier that leverages a pretrained feature extractor and user-provided image templates for each category. Given an input image, the class probabilities are computed by assessing the similarity between the input and the templates. Empirically, the main challenge is distinguishing between 2D orientations (i.c., "left-middie "left-middlee-ightt) and classifying textures (i.e., "front-back"
To address this, we compute the overall similarity by combining orientation similarity and texture similarity in differential "and-gate" manner. The pipeline of classifier is shown in Fig, 2 Drawing inspiration from dense matching techniques (Zhang et al. [024a]b), we propose using a patch-matching distance metric evaluate orientation similarity. Texture similarity is determined by

---

Published as a conference paper at ICLR 2025
Algorithm Uniform Score Distillation
Require: A pretrained diffusion model Epretrains noise predictor cs with optimizable parameters
Ø. set of particles {0' (o=-- text prompt learning rates ! and 72. a rectify function rg and classifier Pę (c(C) wt y) parameterized by &, the number of discrete pose categories ng, the number of time steps vig, EMA update rate Otama-
Initialize the EMA probabilities (Pe(C)))) with (:(cly) = 1/ne.
1: while not converged do
2: Randomly sample {0' Hi=o and c, render the image vo - g(0, c).
3: Apply forward step (xx la, I
4: 0 - 0 Br,ec
(t) (Creetraiu (Ii, t, v) ca (a,, c),) 0g(8.o) 00
+m Bi,
(x)) EL Ve log rgggggg((F11))
5: (r(() + aemaPe (C y) (1 aema p.
6:
$ - Φ - 72 V Ee, (T,, t, c, y) -112-
7: end while
8: return
calculating the cosine similarity of the (cls) token between the input and template images. Orienta- tion and texture similarities are then multiplied after normalization. Finally, the combined similarity normalized using a low-temperature softmax function (Goodfellow al. 2016). For more details on the patch-matching distance and the architecture, please refer to Appendix B.
p(c),t, y) and pe (cly).. By establishing the calculation of p(c|æo, y) with plug-and- play pose classifier, we can now introduce the computation of p(clær. 2) and (cly). To compute p(c|ær, y) J p( 2t, y)p(c//0. y)dx,, we follow DPS (Chung al. 2022) by replacing the 3) and further approximating the expectation with Tweedie's formula (Robbins) 1992). Formally, p(c]æ,,)) v p(c|@p, v), where provide on-the-fly esti-
Estimating
calculation of probability with expectation
2oo (ac, Cppprrraii (a:,, L, y)) //v- Beyond this approximation, mate of the marginal density ps (c(cl). avoiding any form of distribution estimation (Robert 1999). Coneretely, since (c(c)) is the expected value p(c|2x,)) y) over 2t, we update distribution (Cg) using exponential moving average (EMA) of p(c|æ;, y) during optimization, with an update rate Gemas to approximate p(c xt, y). To enable the in-time estimate of the current pose distribution, we propose a time-interval EMA to capture the distribution. Technical details are left in Appendix [B2
The proposed scheme allows for the accurate estimation of the auxiliary function re. facilitating the adjustment of the initial distribution so that the sampling results align with the assumption of uni- form pose distribution. The implementation of uniform score distillation is presented in Algorithm and we refer to this systematic approach RecDreamer.
4 EXPERIMENTS
4.1 EXPERIMENT SETTINGS
To evaluate the performance of USD, we selected 22 prompts describing various objects for compar- ison experiments. The comparison involves three baseline methods (SDS (Poole et al. 2022). SDS-
Bridge (McAlliste al.) 2024). and VSD (Wang et al. 2024b)), and three open- designed to address the Multi-Face Face Janus problem (PerpNeg (Armandpour et al. 2023). Debiased- SDS (Hong et al. 2023). and ESD (Wang et al. (024a) ). We introduce several metrics to assess both the quality of the generated outputs and the severity of the Multi-Face Janus problem. For VSD and USD, we optimize a single particle (i.e., a 3D representation (Mildenhall et al. 202T Müller et al. 2 for score distillation. Additionally, for each prompt, to ensure the text-to-image distribution includes the side and back sub-distributions, satisfying the assumption that p(c) > ( in Lemma (see Appendix B.3 for more discussion).
methods
include auxiliary descriptions
4.2 METRICS
We evaluate our approach through three complementary metrics. The Fréchet Inception Dis- tance (Heusel al. 2017) assesses generation fidelity, while categorical entropy measures quantify
7

---

Published as a conference paper at ICLR 2025
Quantitative comparison. The best and second-best results are highlighted in bold and underlined, respectively. While these metrics provide valuable insights, they may not fully capture all performance aspects. For comprehensive evaluation, please refer the qualitative comparisons
Table 1:
and additional experiments in Appendix C
Method
FID (L)
uFID ())
cEnt (1)
pEnt (1)
CLIP ())
SDS (Poole et al. 2022)
204.81
205.66
1.0235
1.1542
0.6966
Debiased-SDS (Hong et al.
2023
219.46
218.83
1.0171
1.0609
0.7251
PerpNeg (Armandpour et
2023
203.01
203,45
1.0348
1.0390
0.7076
ESD (Wang et al. 2024a)
187.31
188.13
1.0271
1.0928
0.6871
SDS- Bridge (McA llster
2024
230.87
229.41
1.0278
1.0932
0.7250
VSD (Wang et al. 2024b)
168.19
169.66
1.0276
1.0676
0.6807
USD
165.97
165.25
1.0375
1,2488
0.6842
distributional bias. Additionally, CLIP (Radford et al.)
2021) scores measure the alignment between generated scenes and their corresponding text prompts. Details are left in Appendix C.
distribution pairs. We compute standard FID against a base diffusion model (60 images per prompt) and unbiased FID (uFID in Table against its pose-balancee balanced version (by annotating and resampling the generated images). For each method, we render 5 images scene from uniform viewpoints to form the rendered image set for evaluation.
Fréchet Inception Distance (FID). FID evaluates generation quality by comparing
Categorical Entropy. We evaluate 3D consistency by quantifying the Multi-Face Janus Problem through classifier predictions. Inconsistent scenes show similar classification probabilities across viewpoints with bias toward canonical poses, while consistent scenes produce diverse viewpoint- dependent probabilities. We measure this using the entropy of averaged classification probabilities, with higher entropy indicating better consistency. We use two methods: a CLIP-base classifier with directional text descriptions and our proposed pose classifier. The metrics are marked as "cEnt" and "pEnt" in Table [1] For each method, we render 10 images per from uniform viewpoints to calculate the entropy.
CLIP Score. Following ESD (Wang al. 2024a), evaluate text-image alignment by computing CLIP scores between text descriptions and the scenes' corresponding rendered images.
4.3 COMPARISONS
Quantitative Evaluation. As shown in Table our method outperforms other baselines concern- ing the measures for generation quality. However, the limited test set size and comparable texture quality between our method and VSD make challenging to fully quantify the impact geometric consistency through these metrics alone. We provide more details in qualitative comparison and Appendix
C In terms of diversity measures, our method achieves higher entropy scores in both CLIP-based categorization (cEnt) and pose classification (pEnt), indicating that USD effectively incorporates multi-view information into the 3D representations and mitigates the issue of repeti- tive patterns across different viewpoints. Regarding text-image alignment, USD shows lower CLIP scores than VSD, as our multi-view approach incorporates back and side views that may not align with prompts describing predominantly frontal features (e.g., back views of dog versus front- oriented descriptions).
Qualitative Evaluation. As shown in Fig. 3 the texture quality achieved by our method is compa- rable to that of VSD. In terms of geometry, USD demonstrates reasonable structure, capturing the shapes of different poses and successfully simulating some finer details like bumps. Although some artifacts remain (not as smooth as SDS and its variants), our method maintains relatively accurate geometry compared VSD. More results are available in /tttttt hhtp://haaaae05555 thub. La/RecDreemer_proo/]
4.4 ABLATIONS
We provide ablation results in Fig. Since the first stage training establishes the overall geometry, all subsequent experiments are conducted using only the first stage for comparison.
8

---

Published as a conference paper at ICLR 2025
"A DSLR photo of beagle in detective's outfit."
"A portrait of Groot, head, HDR, photorealistic, 8K."
"A kangaroo wearing boxing gloves."
"A DSLR photo of squirrel playing guitar."
(a) SDS
(b) Debiased-SDS
(c) PerpNeg
(d) VSD
(e) USD
Figure 3: Qualitative comparison. The text-to-3D generation results are visualized from three per- spectives (front, left, and right side views), illustrating how the rectified distribution in USD framework effectively mitigates the Multi-Face Janus phenomenon.
Ablation for p(c y). To verify the approximation of p(c|æ,, y), we design a variant that directly estimates the distribution of the noisy images using pose classifier. As shown in Fig. the rectification term this implementation determine the class of noisy images. This causes the rectification work only in relatively small time steps for clean images, limiting its impact on the global optimization. This result highlights the necessity of approximating p(clar,,)) v).
almost ineffective because the classifier struggles to
Ablation for p(cly). Furthermore, to validate the effectiveness of p(cly), we devise a sampling variant. Instead of predicting the current distribution in real-time using EMA, we sample batch of images before training and predict the distribution for each interval. Score distillation is then performed based on this fixed pose distribution. However, the results are quite random (the "bear" case is over-rectified and the "zombie" case is under-rectified). This is because there may be gap between the distribution of score distillation and the sampled distribution, leading guidance to generate another bias. Additionally, fixed rectified distribution is unable to adaptively balance the gradient the noise predictor and the classifier, therefore may lead to over-adjustment.
incorrect
In conclusion, we adopt the approximation p(c|at, y) p(c|ão, y) and employ dynamic distribution updates via EMA. This ensures an effective simulation of the rectification function values.
4.5 COMPONENT ANALYSIS
Hyperparameter Evaluation. We analyze the impact of key hyperparameters, such the update rate of EMA Oema and the number of particles n,, The results indicate that larger Oama facili- tates more responsive updates, allowing for real-time tracking and adjustment of the pose distribu- tion. Additionally, our findings suggest that the back-and-ford time scheduling, as detailed in Ap- pendix B.3 enhances multi-partin optimization. Further specifies can be found in Appendix C.2
Additional Experiments. addition to the hyperparameter evaluation, we conduct further investi- gations, detailed in Appendices C and E Using the annotated pose data, quantitatively val- idate the effectiveness of the pose classifier, with ablation studies on texture and orientation scores confirming the robustness of the classifier architecture. Validation experiments 2D particles pro-

---

Published as a conference paper at ICLR 2025
"Samurai koala bear."
Sampling p(e p(caar at v)
Full
"A zombie bust."
Sampling p(elar,, v)
Full
Figure Ablation studies. We construct two variants, p(c|æ,, y) and Pe(cly) for comparison. The variant p(c|2,, y) directly predicts the category of noisy images It, while the sampling-bar based method, pt (cly), estimates the pose distribution by generating multiple samples and predicting the respective categories.
vide intuitive demonstration of USD's performance. Furthermore, by utilizing RecDreamer, we extend the conditional image generation (Graikos et al. (2022) from one single particle into a multi- particle optimization scheme, enabling more effective control with promising practical applications.
CONCLUSION
In this paper, we presented RecDreamer, novel approach mitigating the Multi-Face Janus prob- lem in text-to-3D generation. Our solution introduces a rectification function to modify the prior distribution, ensuring that the resulting joint distribution achieves uniformity across poses. By ex- pressing the modified data distribution the product of the original density and the rectification function, we seamlessly integrate this adjustment into the score distillation algorithm. This allows us derive particle optimization framework for uniform score distillation. Additionally, we de- veloped pose classifier and implemented reliable approximations and simulations to enhance particle optimization process. Extensive experiments both 2D and 3D synthesis tasks demon- strate the effectiveness of our approach addressing the Multi-Face Janus problem, resulting in more consistent geometries and textures across different views.
Limitations. While our method significantly reduces bias in prior distributions, further exploration of 3D modeling with multi-vies priors could improve geometric and texture consistency. Extending our approach through deeper research into conditional control presents another promising avenue for addressing these challenges in future work.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
The work supported by Guangdong Provincial Natural Science Foundation for Outstanding Youth Team Project (No. 15144151)000000, NSFC Key Project (No. U23A20391), China National Key R&D Program (Grant No. 2023YFE0202700, 2024YFB4709200), Key-Area Research and Devel- opment Program of Guangzhou City (No. 202206030007, 2023B01J0022), the Guangdong Natural Science Funds for Distinguished Young Scholars (No. 2023B 151 55,000977), the Singapore Pro- gramme under the National Research Foundation Singapore (No. AISG3-GVV-2033-01)) I), and the Lee Kong Chian Fellowships.
REFERENCES
Hervé Abdi and Lynne J Williams. Principal component analysis. Wiley interdisciplinary reviews: computational statistics, 2(4):433-459, 2010.
Mohammadreza Armandpour, Ali Sadeghian, Huangjie Zheng, Amir Sadeghian, and Mingyuan Zhou. Re-imagin the negative prompt algorithm: Transform 2d diffusion into 3d, alleviate janus problem and beyond. arXiv preprint arXiv:2304. 04966, 2023.
Yogesh Balaji, Scungjun Nah, Xun Huang, Arash Vahdat, Jiaming Song, Qinsheng Zhang, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, al. ediff-i: Text-to-image diffusion models with an ensemble of expert denoisers. arXiv preprint arXiv:2211.01324 2022.
Changyou Chen, Ruiyi Zhang, Wenlin Wang, Bai Li, and Liqun Chen. unified particle- optimization framework for scalable bayesian sampling. arXiv preprint / 1805. 1659, 2018.
Luxi Chen, Zhengyi Wang, Chongxuan Li, Tingting Gao, Hang Su, and Jun Zhu. Microdreamer: Zero-shot 3d generation in 20 seconds by score-based iterative reconstruction. arXiv preprint arXiv:2404.19525, 2024.
Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fantasia3d: Disentangling geometry and appearance for high-quality text-to-3d content creation. In Proceedings of the IÉEE/CVF inter- national conference on computer vision, pp. 2224622222222 22256, 2023.
Hyungjin Chung, Jeongsol Kim, Michael T Mccann, Mare Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. arXiv preprint wrXiv:2209.14688 2022.
Timothée Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers. arXiv preprint arXiv:2309. 16588, 2023.
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of anno- tated 3d objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13142-13153, 2023.
Jun Gao, Tianchang Shen, Zian Wang, Wenzheng Chen, Kangxue Yin, Daiqing Li, Or Litany, Zan Gojeic, and Sanja Fidler. Get3d: A generative model of high quality 3d textured shapes learned from images. Advances In Neural Information Processing Systems,
2022.
Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning, volume MIT Press, 2016.
Alexandros Graikos, Nikolay Malkin, Nebojsa Jojic, and Dimitris Samaras. Diffusion models as plug-and-play priors. Advances in Neural Information Processing Systems, 35:14715-14728, 2022.
Yuan-Chen Guo, Ying- Tian Liu, Ruizhi Shao, Christian Laforte, Vikram Voleti, Guan Luo, Chia- Hao Chen, Zi-Xin Zou, Chen Wang. Yan-Pei Cao, and Hang-Hai Zhang. threestudio: A unified framework for 3d content generation. threestudio 2023.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems, 30, 2017.

---

Published as a conference paper at ICLR 2025
Jonathan Ho and Tim Salimans. 1rXi::2207.129900 2022.
Classifier-fre
diffusion
guidance.
arXiv preprint
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-655 6851, 2020.
Susung Hong, Donghoon Ahn, and Seungryong Kim. Debiasing scores and prompts of 2d diffusion for view-consistent rext-to-3d generation. Advances in Neural Information Processing Systems, 36:1 1970-11987, 2023.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09683 2021.
Tianyu Huang, Yihan Zeng, Zhilu Zhang, Wan Xu, Hang Xu, Songcen Xu, Rynson WH Lau, and Wangmeng Zuo. Dreamcontrol: Control-based text-to-3d generation with 3d self-prior. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5364-
5373, 2024.
Yukun Huang, Jianan Wang. Yukai Shi, Xianbiao Qi, Zheng-Jun Zha, and Lei Zhang. Dream- time: An improved optimization strategy for text-to-3d content creation. arXiv preprint arXiv:2306. 12422, 2023.
Chenhan Jiang, Yihan Zeng, Tianyang Hu, Songeun Xu, Wei Zhang, Hang Xu, and Yan Yeung. Jointdreamer: Ensuring geometry consistency and text congruence in text-to-3d generation via joint score distillation. In European Conference on Computer Vision, pp. 439-456. Springer, 2025.
Yutao Jiang, Yang Zhou, Yuan Liang, Wenxi Liu, Jianbo Jiao, Yuhui Quan, and Shengfeng He. Diffuse3d: Wide-angle 3d photography via bilateral diffusion. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8998 9008,900, 2023.
Heewoo Jun and Alex Nichol. Shap-e: Generating conditional 3d implicit functions. arXiv preprint arXiv:2305. 02463, 0200:2335500463,
2023.
Oren Katzir, Or Patashnik, Daniel Cohen-Or, and Dani Lischinski. Noise-free score distillation. arXiv preprint arXiv:2310.17590. 2023.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis. 3d gaussian splat-
ting for real-time radiance field rendering. ACM Trans. Graph., 12 2023.
Solomon Kullback and Richard A Leibler. On information and sufficiency. The annals of mathe- matical statistics, 22(1):79-86 86, 1951.
Yixun Liang, Xin Yang, Jiantao Lin, Haodong Li, Xiaogang Xu, and Yingcong Chen. Luciddreamer: Towards high-fidelity text-to-3d generation via interval score matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6517-6526, 2024.
Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Yu Liu, and Tsung-Yi Lin. Magic3d: High-resolution text-to-3d con- tent creation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 300-309, 2023.
Haofeng Liu, Chenshu Xu, Yifei Yang. Lihua Zeng, and Shengfeng He. Drag your noise: Inter- active point-based editing via diffusion semantic propagation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 6743-6752. 2024.
Qiang Liu and Dilin Wang. Stein variational gradient descent: general purpose bayesian inference algorithm. Advances in neural information processing systems, 29, 2016.
Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick. Cero-I-to-3: Zero-shot one image to 3d object. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 2299-9309, 2023.
12

---

Published as a conference paper at ICLR 2025
Baorui Ma, Haoge Deng, Junsheng Zhou, Yu-Shen Liu, Tiejun Huang, and Xinlong Wang. Geo- dream: Disentangling 2d and geometric priors for high- figh-fidelity and consistent 3d generation. arXiv preprint arXiv:2311.179999 2023.
David McAllister, Songwei Ge, Jia-Bin Huang, David W Jacobs. Alexei A Efros, Aleksander Holyn- ski, and Angjoo Kanazawa. Rethinking score distillation as a bridge between image distributions. arXiv preprint (:XXi:2406.09// 2024.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes neural radiance fields for view synthesis. Communications of the ACM, 65(1):99-100 2021.
Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics prim- itives with a multiresolution hash encoding. ACM transactions on graphics (TOG), 41(4):1-15, 2022.
Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2 10741, 2021.
Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen. Point-e: A system for generating 3d point clouds from complex prompts. arXiv preprint arXiv:22/2.08751, 2022.
United States. National Bureau of Standards and Fred Edwin Nicodemus, Geometrical consider- ations and nomenclature for reflectance, volume 160. US Department of Commerce, National Bureau of Standards Washington, DC, USA, 1977.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023.
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d
diffusion. arXiv preprint arXiv:2209. 14988, 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp. 8748-8763, PMLR, 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.06122 1(2):3, 2022.
Herbert E Robbins. An empirical bayes approach to statistics. In Breakthroughs in Statistics: Foundations and basic theory, pp. 388-394. Springer, 1992.
CP Robert. Monte carlo statistical methods, 1999.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomed- ical image segmentation. In Medical image computing and computer-ussisted intervention- MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceed- ings. part III 18, pp. 234-241. Springer, 2015.
Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d generation. arXiv preprint arXiv:2308. 16512, 2023.
Yawar Siddiqui, Antonio Alliegro, Alexey Artemov, Tatiana Tommasi, Daniele Sirigatti, Vladislav Rosov, Angela Dai, and Matthias Nießner. Meshgpt: Generating triangle meshes with decoder- only transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattrnn Recognition, pp. 19615-19625 2024.
13

---

Published as a conference paper at ICLR 2025
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learn- ing. pp. 2256-2265 PMLR, 2015.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint AxXiv:2011.13456: 2020.
Jingxiang Sun, Bo Zhang, Ruizhi Shao, Lizhen Wang, Wen Liu, Zhenda Xie, and Yebin Liu. Dreamcraft3d: Hierarchical 3d generation with bootstrapped diffusion prior. arXiv preprint arXiv:2310. 16811888 2023.
Haochen Wang. Xiaodan Du, Jiahao Li, Raymond A Yeh, and Greg Shakhnarovich. Score jaco- bian chaining: Lifting pretrained 2d diffusion models for 3d generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12619-12629, 2023a.
Peihao Wang, Zhiwen Fan, Dejia Xu, Dilin Wang, Sreyas Mohan, Forrest landola, Rakesh Ranjan, Yilei Li, Qiang Liu, Zhangyang Wang, et al. Steindreamer: Variance reduction for text-to-3d score distillation via stein identity. arXiv preprint arXiv:2401.0066 2023b.
Peihao Wang, Dejia Xu, Zhiwen Fan, Dilin Wang, Sreyas Mohan, Forrest landola, Rakesh Ranjan, Yilei Li, Qiang Liu, Zhangyang Wang, et al. Taming mode collapse in score distillation for text- to-3d generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9037-9047, 2024a.
Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Pro- lifiedreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. Advances in Neural Information Processing Systems, 36, 2024b.
Chenshu Xu, Yangyang Xu, Huaidong Zhang, Xuemiao Xu, and Shengfeng He. Dreamanime: Learning style-identity textual disentanglement for anime and beyond. IEEE Transactions on Visualization and Computer Graphics, 2024.
Taoran Yi, Jiemin Fang, Guanjun Wu, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Qi Tian, and Xinggang Wang. Gaussiandreamer: Fast generation from text to 3d gaussian splatting with point cloud priors arXiv preprint ar arXiv:2310.0852: 2023.
Xin Yu, Yuan-Chen Guo, Yangguang Li, Ding Liang, Song-Hai Zhang, and Xiaojuan Qi. Text-to-3d with classifier score distillation. arXiv preprint arXiv:2310. 19415, 2023.
Yuyang Yu, Bangzhen Liu, Chenxi Zheng, Xuemiao Xu, Huaidong Zhang, and Shengfeng He. Be- yond textual constraints: Learning novel diffusion conditions with fewer examples. In Proceed- ings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 7109-7118, 2024.
Junyi Zhang, Charles Herrmann, Junhwa Hur, Eric Chen, Varun Jampani, Deqing Sun, and Ming- Hsuan Yang. Telling left from right: Identifying geometry-aware semantic correspondence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattem Recognition, pp- 3076-3085, 2024a.
Junyi Zhang, Charles Herrmann, Junhwa Hur, Luisa Polania Cabrera, Varun Jampani, Deqing Sun, and Ming-Hsuan Yang. A tale of two features: Stable diffusion complements dino for zero-shot semantic correspondence. Advances in Neural Information Processing Systems, 36, 2024b.
Longwen Zhang, Ziyu Wang, Qixuan Zhang, Qiwei Qiu, Anqi Pang, Haoran Jiang, Wei Yang, Lan Xu, and Jingyi Yu. Clay: A controllable large-scale generative model for creating high-quality 3d assets. ACM Transactions on Graphics (TOG), 43(4):1-20, 2024c.
Chenxi Zheng, Bangzhen Liu, Xuemiao Xu, Huaidong Zhang, and Shengfeng He. Learning an inter- pretable stylized subspace for 3d-aware animatable artforms. IEEE Transactions Visualization and Computer Graphics, 2024.
Junzhe Zhu, Peiye Zhuang, and Sanmi Koyejo. Hifa: High-fidelity text-to-3d generation with ad- vanced diffusion guidance. arXiv preprint ar0iv: /8766, 2023.
14

---

Published as a conference paper at ICLR 2025
A RELATED WORKS
A.1 DIFFUSION MODELS FOR TEXT-TO-IMAGE GENERATION
A major challenge in text-to-image generation using diffusion models Yu et al,, 2024; Liu et al, 2024 Xu al. 2024) is guiding the generative process to reflect the input text accurately. A widely adopted solution is classifier- free guidance (CFG, Ho & Salimans (2022), which eliminates the need for external classifiers by training a unified model for both unconditional and conditional image generation. During inference, CFG achieves conditional generation by interpolating between the conditional and unconditional scores, effectively guiding the model match the input text. This method has shown significant success in various text-to-image tasks (Balaji et al., 2022; Nichol al. 202T, Ramesh et al., 2022). Models like DALL-E and Stable Diffusion (Rombach et al. 2022 have demonstrated exceptional capabilities in producing diverse and complex images, with promising extensions into text-to-3D generation.
A.2 TEXT-TO-3D GENERATION
Recent advances in text-to-3D generation can be broadly divided into two main approaches. The first approach focuses directly learning 3D asset distributions from large-scale datasets such as Objaverse (Deitke et al.; 2023). Notable models within this category include GET3D (Gao et al. 2022). Point-E (Nichol et al. 2022), Shap-E (Jun & Nichol 2023), CLAY (Zhang et al. (024c). and MeshGPT (Siddiqui et al. 2024). all of which leverage extensive 3D data to generate accurate 3D models.
The second approach relies on 2D priors (Jiang et al. 2023) for generating 3D models. Techniques like score distillation are foundational here, as exemph exemphiied by DreamFusion/SJC (Poole al. 2022 Wang et al.
2023a) and ProlifieDreamer (Wang et al. 2024b).
Building on these baselines, researchers continue improve visual quality. Classifier Score Dis- tillation (Yu et al. 2023) reframes the fundamental approach by treating classifier-free guidance the central mechanism rather than supplementary component, enabling
realistic synthe- sis. Noise-Free Score Distillation (Katzir et al.] (023)) addresses the over-smoothing problem by climinating unnecessary noise terms, allowing for effective generation standard guidance scales. SteinDreamer (Wang et al., 2023b) introduces Stein's 's identity to reduce gradient variance in score distillation for faster and higher-quality generation. LucidDreamer (Liang et al., (024) tackles the over-smoothing challenge differently by combining interval score matching with 3D Gaussian Splat- ting (Kerbl et al., 2023) and deterministic diffusion paths. Most recently, SDS-Bridge (McAllister
et al. 2024) enhances the entire pipeline by introducing calibrated sampling based on optimal trans- port theory and improved distribution estimates.
A separate line of research focuses improving geometric quality. Magic3D Lin al. 2023) enhances output resolution by first generating a coarse 3D hashgrid and subsequently refining it into mesh. Fantasia3D (Chen et al., 2023) introduces hybrid scene representations and spatially varying BRDF (of Standards & Nicodemus 1977) for realistic modeling. Other models, such as Dreamtime (Huang al., (003) and HiFA (Zhu et al. (2023), concentrate on optimizing time-step sampling to improve texture stability and geometric consistency.
A.3 ALLEVIATING THE MULTI- FACE JANUS PROBLEM
One of the key challenges when extending text-to-image priors to 3D is the Multi-Face Janus prob- lem, where inconsistencies arise in 3D geometries, especially for objects with multiple faces. To address this, Yi et al. (2023) and Ma et al.] (2023) introduce pretrained shape generators that pro- vide geometric priors. DreamCrafi framework, combining view-dependent diffusion with Bootstrapped Score Distillation to separate geometry and texture optimization. MVDream (Shi et al. 2023) introduces a dedicated multi-view diffusion model that bridges 2D and 3D domains, enabling few-shot learning from 2D examples. JointDreamer (Jiang et al. 2025) presents Joint Score Distillation with view-aware models as energy functions to ensure coherence by explicitly modeling eross- cross-view relationships. While these methods effectively handle diverse scenarios, certain complex text descriptions can still pose challenges due to inherent limitations in pretrained generators and multi-view generative models.
(Sun et al. (2023) tackles the challenge through a hierarchical
15

---

Published as a conference paper at ICLR 2025
Beyond introducing basic geometric priors, several methods have aimed to improve control over pase prompts. Debiased-SDD (Hong et al. (023) tackles text bias by removing words that conflict with pose descriptions, while Perp-Neg (Armandpour et al. 2023 proposes a perpendicular gradient sampling technique to remove undesired attributes from negative prompts. Other works have sought to address pose bias in pretrained models by altering the approximation distribution through pose sampling or entropy constraints.
DreamControl (Huang al. 2024) approaches the pose bias issue by employing adaptive viewpoint sampling, which adjusts the rendering pose distribution to better mimic the inherent biases of the model. Additionally, ESD (Wang et al., 2024a) demonstrates that the score distillation process degenerates into maximum cirimmum-ikeelhhoed likelihood seeking, and proposes an entropic term to introduce diversity across different views, helping to prevent repetitive patterns in 3D generation.
B METHODOLOGICAL DETAILS
In this part, we further complement the introduction the RecDreamer. Appendix B.] presents the detailed architecture of the pose classifier. B.2 provides more detailed derivation of the approxi- mation discussed in Sec. 3.3] and an explanation of the EMA update process. In B.3 we present the precise implementation details of RecDreamer. Lastly, the proof of the main theorems is given in B.4
B. POSE CLASSIFIER
B.1.1 OVERVIEW
The classification of pose typically involves two key points: when shapes of input and template im- ages are similar (front and back), texture information is usually used for differentiation. Conversely, when shapes different (left, center, and right), attention must be given to the features' positions.
As introduced in Sec. 3.3 the classifier mimics an "AND gate" structure, combining texture sim- ilarity and orientational similarity between the input and template for computation. Here, texture similarity mainly distinguishes the front and back of objects, while orientational similarity differen- tiates the three 2D orientations (left, center, and right).
Texture similarity is obtained by calculating the cosine similarity between global features (i.e., [els] token) derived from the input and template images. However, orientational similarity cannot be easily derived from pretrained feature extractors, as they often use image augmentation techniques similar global features for left and right orientations. To address this issue, we propose the most matching patch distance to measure orientational similarity. The overall process illustrated in Fig. and formulation is detailed in Appendix B.1.2
(like flipping) during training, leading
B.1.2 ARCHITECTURE
Assume that we have a feature extractor F(-) that maps an image to the feature space. Given the height h and width of the features,
denote the global feature and patch features of an input image x e Rdd as fels E R and fpat E IR xx i.e., fele fpat = F(x). The features of template images {x' [0 < < np} are given by { 0 < number of the pose categories. To calculate the texture similarity, we directly calculate the cosine similarity cos(-..) between the global input features and the template features as follows:
< np), {f 0 S < np), where np is the
Ster { 10 < <i<in <p))
(15)
Stex cos(fetss fai.).
In Eq. 15
we consolidate the per-class texture similarity scores (scalars) sie, into a vector represen- tation Stez to streamline notation.
To calculate the orientation similarity, we propose the matching patch distance to evaluate the ori- entation discrepancy between the input and output images. To concentrate the main subject, introduce the binary mask (the calculation of binary mask is introduced in Appendix B.133) both input and template images, denoted as b and (b'j0 < < np), where b E Rixaxi, We also
16

---

Published as a conference paper at ICLR 2025
distribute coordinate map for the input and template images based the binary masks mark the relevant coordinate, denoted as m and m'.. To be specified, the leftmost pixel of the subject is assigned with value -0,5 and the rightmost 0.5, where the value of intermediate pixels between the leftmost and rightmost are interpolated from 0.5 to 0.5.
We denote the patch features by fat and fpa.. the binary masks by b and b and the coordinate maps by m and m paa- At specific patch coordinate u (z. y), these quantities are written as (fu: fü), (bu, ba),, and (m,, my), respectively. For subject patches (that is, coordinates where the binary mask is 1), we collect the corresponding features and coordinates into the sets fpat = {ff I by 1) and m {m ba 1}. Similarly, for template images, denote these sets by fine and m'.
For the input image, the similarities between the foreground patch fyar,u and the template patch fpat.w is accessed by calculating the L2-norm, i.e., Su,u' Typae (.) is softmax function with temperature. Then, the mean distance between the input and the template is computed by traversing all the patches of the input image and the template image, and use the negative of the distance indicate the similarity of orientation:
Tpaa (1l fott - - fp l12), where
Sori (sord 1 S Kp),
ΣΣ
uu, [i mi
(16)
| |u/l|2/||
a²
Finally, we compute the final probability pooee by:
8 pose = O Tposs (Stez Sori).
(17)
Here, Tpoosc (-) is softmax function and @ signifies element-wise multiplication. In Fig. 5 we demonstrate the distance calculation between a single pixel u from the input image and the pixels {u'} in the templates.
B.1.3 FOREGROUND MASK SEGMENTATION
To segment the foreground subject, we follow DINOv2 (Oquab et al.) 2023 Darcet al. 2023) and apply a Principle Component Analysis (PCA) (Abdi & Williams/ 2010) to the patch features (f < np)) The PCA algorithm reduces the dimensions of the patch from R wx to
Rhxwxf' where f' is a small number. We use the first component along the feature channel dimen- sion, the basis for foreground segmentation. However, since it's unclear whether 0 corresponds to the subject or < 0 does after PCA, we introduce the cross-attention feature map from Stable Dif- fusion (Rombach et al. 2022) for guidance. Concretely, we encode the template image into latent space and use the cross-attention features of the image and prompt at different time steps to obtain the subject area. Subsequently, based on the cross-attention feature map, we can infer whether the positive or negative value of PCA's first component stands for foreground. Note that we do not use cross-attention directly as segmentation map because this method more complex and not suit- able for efficient segmentation during training. Instead, using PCA for segmentation is lightweight approach that does not affect training efficiency.
1.1.4 IMPLEMENTATION OF POSE CLASSIFIER
To minimize the impact on efficiency, we employ the feature extractor "dinov2.vits14" We resize the classifier features to a size h = 16. The temperature for patch-wise similarity Tpat is set to 0.01, while the temperature for pose in Eq. 16 is set to 0.05 to enhance the distinction between categories. When implementing the classifier (calculating PCA parameters), tations (random noise, random affine, random grayscale, and color jitter) to the template images to achieve robust sregrrouddbbackroound segmentation and classification.
apply data augmen-
17

---

Published as a conference paper at ICLR 2025
-0.
455
0.5
0.5
(a) a
(b) b
(c) 77
(d) my
Suct m. 103
+0.5
-0.2
0.2
0.5
0.11
0.73
-073x
+003* 0-05 0:22
-02
+0.5 X 1-0.5 0.21
40.1 x 1-0.5 0.2
0.5
+0.
0.2
0.5
0
0.943
(c) x2
6
772
(h) Su.w! for m
(i) mean distance
0.2
0.03
Fixs M, -3
0.5
1 x ( (.5))
+008 0-55 (-5))
+8.3 x 0-.5- (-02)
4.2
0.2
+0.05
(nIEX 0.5 (-02)
1 (-02)
0.5
(j) ta
(k) t
r
(m) Su,w m is (n) mean distance
Figure 5: Calculation of orientation distance. (a) is the input image for testing. (e) and (j) are tem- plates showing "facing right" and "facing left" orientations. Each row demonstrates the orientation distance between the specific image patch (d) and the templates. For opposite orientations, the mean distance is larger compared to the similar orientations (i) v.s. (n)). This feature is important for the classification of pose.
B.2 CALCULATION THE RECTIFICATION FUNCTION
Approximation of p(c| v). In Sec. 3.3 we have demonstrated the approximation of p(c xe v)- By expanding the derivation of Eq. 7 of DPS (Chung al., 2022) conditional form, we have:
/ p( we, ()p(c Tt y)
p(c (e
(18)
= p(an ær, y)p(cl/a xp. daoo
w p(claro,
Note that we follow DPS's assumption that pose c independent to T.. Eq. 18 approximated by exchanging the calculation of expectation and p(c|æu, y). i.e., p(c|2,, v) ~ p(c/Co. v), where E Epp (n [zo]] By Tweedie's formula, ty (xt dtCrettroin (Ze. t.))) /aa))
Estimate of pe(cly). Another term, (cly)), represents the expectation of p(c|æ,, y) a.. To estimate Pr (cly), we employ an exponential moving average (EMA) to iteratively update this term with the values of p(c 2 y) during training. Since (c(yl)) is time-dependent, each time step in the diffusion model requires a corresponding EMA value. However, updating the EMA at each iteration only occurs with a probability of 1/1000, which does not accurately track the current distribution. Fortunately. we observe that the empirical discrete pose probability of Pt (cly)) for adjacent time steps are nearly identical (e.g., g., pı (Ely) and p2(Ely) are almost the same). values across multiple time steps can be unified within intervals to enhance efficiency. We denote the number of intervals as n4, and the number of steps within each interval, n.. is calculated as n. = T/m,, where T is the training step of DDPM. We maintain list of EMA values for different intervals, E 1n 1==0- The EMA version the pose probability, Pe (c y). then given by Pa (cly)) =
a result, the EMA
== l/n.] The update of the EMA follows the update rate Gemas as described in the following: Vemm
18

---

Published as a conference paper at ICLR 2025
Pe(Cly) < CemaPt (c/0e.y) + +(1 Otema (c(3))
(19)
Empirically, given 7= 1000 is the total steps, we set n4 = 10 and n1 100. Oema is set to satisfy that the previous Reme samples has total EMA weights greater than 0.9.
B.3 OTHER IMPLEMENTATIONS
Additionally, we implement other tricks to ensure the effectiveness of gradient updates at different time steps.
Gradient norm. The optimization dynamics of uniform score distillation present fundamental challenge: the framework simultaneously processes two conflicting gradient signals (classifier gra- dients and denoiser gradients) with inherent magnitude disparity. Through empirical measurement, we observe that classifier gradients for rendered images 2y exhibit substantially higher magnitudes compared to denoiser gradients. To resolve this optimization instability, we implement gradient norm alignment- specifically constraining the L2 norm of the classifier gradient to match the de- noiser gradient norm. This normalization strategy ensures balanced parameter updates while pre- serving relative gradient directions.
Time scheduler. Furthermore, utilized back-and-forth (BNF) time scheduler. The time sched- uler constrains the sampling interval for each iteration. Assuming the number intervals for BNF denoted ni, we divide the total number of iterations into 2n, intervals. For the first n, intervals, the sampled time steps are expanded from [T's * 0.98, T -T//x) to [T' + 0.98, T * 0.02]. For the last n, intervals, the sampled time steps are reduced from [T * 0.98, T 0.02] to ((T/m)), T 0.02). Typically, we set = 2 for one particle optimization.
Three-stage optimization. Similar to VSD, we use three-stage optimization paradigm. For the first stage, we train the Instant-NGP (Müller et al. 2022) using USD for 15k iters. In the second stage, we use SDS for geometric refinement for 15k iters. the third stage, we optimize the texture with USD for 15k iters.
Auxiliary prompts. Our method is essentially reweighting the subdistribution of prior distribu- tion, so in the proof of Lemma [ we assume that p(c) 0. To ensure this condition holds, we augment the original prompt with phrases like "from side view, from back view." to introduce additional pose information. While these auxiliary prompts may not create perfectly balanced dis- tribution, they guarantee that p(c) > by incorporating multiple viewpoints. Our algorithm then adjusts this distribution achieve uniformity. It is important to note that differs fundamentally from prior work. Previous research has primarily employed directional text for conditional generation, aiming constrain the model to produce content that strictly aligns with the provided text. In contrast, our approach leverages directional prompts to expand the model's distribution, allowing it to capture more diverse range of information.
of directional text
B.4 PROOF OF MAIN THEOREM
B.4.1 PROOF OF THEOREM I
Since two marginal distributions p(æ) and p(c) are involved, we first study their joint distribution p(æ,c) We introduce the weighting function w(c) to correct p(æ, c) so that the rectified marginal distribution obeys the target distribution fc). The rectified joint distribution given by the follow- ing lemma
Lemma 1. Given the original joint distribution p(æ, c), where p(c) is the marginal distribution of
c, and p(c) + O, and larget marginal distribution [(c), we can rectify p(c) to f(c) by introducing a weighting function w(c) = p(c)*
f(e). The corrected joint distribution p(a, c) is then given by:
p(x, c) = w())p(x, c) = f(c) p(c) pla,
(20)
Proof of Lemma 7
To adjust the marginal distribution p(c) to the target distribution f(c), we apply weighting function wo(c) to the original joint distribution following importance sampling. The new
19

---

Published as a conference paper at ICLR 2025
joint density is given by:
p(x,e) e) = w(c)p(a, c).
(21)
The marginal distribution of under j(ær, c) is:
p(c) = p(z, c)dx==)/x/---
w(c)p(c).
(22)
Substituting this into the expression for p(æ, c) gives Eq. 20 Since [(c), p(c), and p(a, are non-negative, ((x,c) 2 0. To validate normalization, we compute:
To satisfy p(c) = f(c), we set w(c)
ple)"
I Iʃ p(æ, c(æ,c)ddde = / (f p(a)p(c(a))))) de 1. p(c)
f(e)
(23)
This confirms that p(æ, e) is a valid probability distribution, which completes the proof.
Below provide proof of Theorem
Proof of Theorem According Lemma I the rectified joint distribution p(x, c) satisfies that the marginal distribution p(c) = f(c). The rectified data density p(x) is obtained by marginalizing p(x,,) c) over as follow:
plw) p(z) / p(c)
f(c)
f(c) p(c) p(cl))dde
p(x) = p(a.
p(x,c)dc- c)dc
(24)
This completes the derivation of p(æ)).
B.4.2 COROLLARY OF THEOREM
We generalize the conclusions of Theorem to conditional distributions as follows.
Corollary 2. For the conditional case, we extend the result p(x=ly) as:
/
[(cyy p(c]x, y).c. p(cly)
p(zly) = p(xlv)
(25)
This follows directly from the general form by conditioning on y.
Proof of Corollary
2 Analogous to Lemma I we aim to rectify the conditional marginal distribu-
tion p(cly) the target distribution f(lly). By p(cly) J plc) w(cly)p(cl)) f(ly),
we derive w(ccy) = ficly). The rectified distribution p(x: y) is then expressed by integrating over c p(clo)
as follows:
blc, - /
plc, aly)de p(arl)) / p(cly)
f(cly)
f(cly) p(c|2r y)de.
p(xly) =
p(cly) p(cyyp
(26)
This completes the derivation of p(x:ly)
B.4.3 PROOF OF THEOREM 2
We provide the proof for deriving the rectified density for different time steps.
Remark 1. Our primary objective is to obtain the rectified density t= imposing a uniform distribution across all other noise states. Consequently, for any >> 0, the density (a(w)))) should be derived from transition from (oool))) as detailed in the following proof.
= O, i.e., Po I v), without
Proof of Theorem First, we consider the f = 0. According to Corollary 2 we have:
/
f(cly)
Po( = (o(
p p(c|20, y)dc.
(27)
po(cly)
20

---

Published as a conference paper at ICLR 2025
For any t E [1, the probability of noisy images is given by:
/ po (2oll))) )
(e(2)ll))
/ V I ((cl))((0olc udee pro (ov (20 deoo
(
f f(cl)) / p(zole, v) (2 (2)) dc f f(cl))
(
/ placole u)pro lo. e, y) y)dwo de
(28)
/ f(cly)
/ plzx,, xule v)dwo dc
/ f(clu)p(xrlc, y)dc
p.((a.l)) /
(cll)) (clc)y) p(c|ar, y)de,
where (a) is according to Fubini's theorem, (b) based the that the forward process proceeds according to the original scheme, unaffected by text or pose conditions. By combining Eq. 27 and Eq. 28,
conclude that for any E [0, [). Eq. holds, completing the proof.
B.4.4 PROOF OF COROLLARY 1
This corollary directly follows from Theorem 2 as proposed by VSD WWang et al. 2024b).
Proof. Theorem by VSD establishes that the update rule for each particle 0, at ODE time within a Wasserstein gradient flow is given by:
d0, Er, (owllt) (Vx, (r (2a) Va, log पt" (xxle,
cg(O,.c)
(29)
dr
00,
In the of rectified distribution, the update rule is modified as follows:
do, Ex,,cc (ww((( (V., log (o V.. log qt (x.(c,))) Og(0,,
(30)
dr
00,
which can be further simplified as:
d0,
0g(0,, c)
Eree (tw(t) (Vz, log (((eel))ra ne, log ttth (xzle, v))
dr
80,
0g(0r, c)
= -E,,c (lw 2V logr(#:/u) Va, log (æ. V n, log logr(x:(y))
00,
0g(0,.c) w(t) ot -2 Vo. logr(x):)))
E,, w(t) (E,, c, Eprctrain (x, t, v))
00,
ar
(31)
Therefore, g(0) can be update by g(i) +- g(i) 1/ (VoLusD(((1)) where:
VoLUSD = VoL's Er,,c w(()) a: Vo Pologr(zzilb)
ar
(32)
Og 0g c)
VoL'VD = Erec w(()) (Epretrain (2,,, 1. -
( 1, c.
00
Proof complete.
21

---

Published as a conference paper at ICLR 2025
C SUPPLEMENTARY EXPERIMENTS
This appendix contains supplementary experimental details. C.1 provides a detailed discussion of metric calculations, C.2 analyzes the influence of hyperparamelers. e
visualizes the user-provided templates. C.4 demonstrates our method's scalability through cross-domain rectification. C.5 ex- plores special cases to showease practical upplications.(..0 [C6 [C.7 results. Note that the prompt list, comparisons, and results are left in Appendix GaaiiiixGaann and Appendix
present additional performance
C.1 METRICS
C. FRÉCHET INCEPTION DISTANCE FOR GENERATION QUALITY
The Fréchet Inception Distance (Heusel al. 2017) (FID) serves as our primary metric for evaluat- ing generation quality by measuring the statistical distance between two image distributions. In our evaluation process, we compare our generated images against two different target distributions:
Standard FID.
To evaluate the quality gap between our generated 3D scenes and pretrained Stable Diffusion (Rombach et al. 2022) outputs, we establish a target distribution by sampling 60 images per prompt across 22 different prompts, yielding total test set of 1,320 images. To mitigate pose bias in this distribution, we incorporate directional text descriptions such as "front view," "side view," and "back view" during sampling. However, we note that some pose bias remains, with frontal views being over-synthesized. For our generated distribution, we render 5 images from each 3D scene using uniformly sampled camera poses. The standard FID score is then calculated between this rendered set and our target distribution.
Unbiased FID (uFID). To address the inherent pose bias present standard FID evaluation, we develop an alternative metric called uFID. This approach begins with manual annotation of camera poses for all images in the standard test set. Using these annotations, we resample the test set to ensure equal representation across different poses. While this resampling strategy may result in some image duplication, it yields a more balanced distribution of viewpoints and textures. The uFID score is then computed between this pose-balanced dataset and our rendered images, providing a more equitable assessment of generation quality across different viewpoints.
C.1.2 CATEGORIAL ENTROPY FOR GEOMETRIC CONSISTENCY
We evaluate the Multi-Face Janus problem using a pose classifier to analyze viewpoint consistency across different perspectives. The underlying principle is straightforward: in scene with severe Multi-Face Janus issues, different viewpoints will yield similar classification probabilities because they share similar features. This similarity typically manifests as a strong bias toward a particular class (usually the canonical pose) in the average classification probability across viewpoints. Con- versely, a geometrically consistent 3D scene will produce more diverse classification probabilities that average toward a more uniform distribution across viewpoints.
Based on this insight, we use the entropy of the average classification probability from different views as metric for measuring the severity of multiplicity problems. Higher entropy values indicate greater diversity in information across viewpoints, while lower values suggest excessive pattern duplication in the generated scene. Formally, for each prompt, we calculate the entropy Rent as:
p
(g((,.c)) ci)),
Thp
(33)
1 ne nh
Rent
p log P.,
where 9 represents the renderer and & the pose classifier. The probability vector consists of com- ponents p.. ne denotes the number of sampled views (default: 10), and na represents the number of pose categories. We implement this entropy evaluation using two different classification approaches:
CLIP Entropy (cEnt). This method employs CLIP (Radford et al, 2021 as the classifier, us- ing three textual descriptions that combine the original prompt with directional modifiers: "from
22

---

Published as a conference paper at ICLR 2025
"A DSLR photo of beagle in detective's outfit."
10
"A portrait of Groot, head, HDR, photorealistic, 8K."
71
TH = 10
(a) 1t0
1tcrs=15,000
iters=25,00 ,000
Figure 6: Influence of number of BNF intervals for multiple particles. Experiments compare performance with n, = 2 versus n, 10 intervals. The finer granularity of BNF intervals (n, 10) leads to better synchronization between particle generations, mitigating the Multi-Face Janus Problem.
front view", "from side view", and "from back view". These descriptions establish three distinct categories for classifying input images.
Pose Entropy (pEnt). This variant utilizes our specially designed pose classifier for categoriza- tion, providing a more direct assessment of pose-related geometric consistency.
C.1.3 CLIP SCORE FOR TEXTUAL ALIGNMENT.
To evaluate textual alignment, we calculate the CLIP score by measuring the negative cosine similar- ity between CLIP feature embeddings of the rendered images and their corresponding text prompts, following Wang et al. 2024a)..
C.2 HYPERPARAMETER ANALYSIS
Influence of BNF interval n. The BNF time scheduler controls the sampling intervals for score distillation. A larger п, provides finer control over the sampling process, making more closely resemble the DDPM sampling process. Through our single-particle optimization experiments, we observe that adjusting n; values demonstrates minimal influence on training dynamics until the terminal phase of optimization. During this final stage, larger π4 configurations become particularly impactful. For example, when the sampling time step is limited to the interval [100, 0], the model tends to overfit, often producing oversaturated colors. We typically use early stopping to prevent this. However, in simultancous multi malticparticle optimization, we observe that a larger BNF n, improves training quality. As shown in Fig. 6 the training process for different BNF values demonstrates
23

---

Published as a conference paper at ICLR 2025
Mema
10,00 100
0,01
0.80
3300
228
1130
18
1234
224
Rema
1,000
110
Ael right
222
11055
Bors
Itors
11055
Remma 100
baidt
0.2
Figure 7: The impact of varying the number of valid EMA steps 7emo on pose probability distri- butions over time. Each curve shows Pe(āly). representing how pose distributions evolve during training. For visualization clarity, we map the continuous time t to discrete indices t', where each index spans 100 timesteps (e.g., t' corresponds E [0, 100)). lower nema enables timely correction of distribution bias, resulting in more stable probability distributions (i.e., please zoom in a better view of the y-axis).
"A kangaroo wearing boxing gloves."
The = 3
We = 4
Vhc 6
Figure 8: Influence of the number of templates My. Varying na shows minimal impact on perfor- mance, as our pose classifier is designed for coarse categorization and cannot effectively distinguish fine-grained poses at higher values.
this effect. When = 2, the training is imbalanced across particles. For example, in the case of "beagle," the first particle learns the information more quickly, while the fourth particle of "Groot" progresses more slowly. This imbalance causes the fastest particle to converge to one mode of the distribution, such as all back views for "beagle", while other particles converge to different modes (e.g.. front view, back view, etc.). This results in an undesirable outcome, where the overall distribution across particles appears uniform, but each individual particle suffers from the Multi-Face Janus problem-on one is biased toward front views, while another is biased toward back views, which contradicts our goal. To resolve this issue, we ensure that all particles converge at consistent rate. Our BNF time scheduling controls the interval of sampling time steps, where larger BNF value (n,) results in time steps being sampled within narrower range during the early training phase. This promotes uniform optimization process across particles. As result, training consistency
24

---

Published as a conference paper at ICLR 2025
"Samurai koala bear."
USD
(a) po(cly)
(b) Results
Figure 9: Comparison with the q sampling and our USD. q sampling modifies uniform view sam- pling using estimated pose probabilities (cly)) (a) shows the estimated po(cly) distribution for the prompt, with probabilities approximately [0.1, 0.6, 0.15, 0.15) for the front (red), back (blue), left (yellow), and right (green) views. (b) shows the corresponding generation results.
"DSLR Camera, photography, dsIr, camera, noobie, box-modeling, maya."
DSLR photo of a chimpanzee dressed like Napoleon Bonaparte."
"A DSLR photo of squirrel playing guitar."
"A zombie bust."
(a) Stable Diffusion
(b) Zero 1-to-3
Figure 10: Examples of template images. (a) Templates manually curated from Stable Diffu-
sion Rombach et al. 2022) generations. (b) Multi-view images obtained using Zero 1-to-3 (Liu al. 2023). Note that the multi-view image need not be completely 3D consistent and high-quality.
we show the generation effect with n, = 10, which better aligns the modified distribution across all four particles.
improves, reducing the risk of biased convergence. In Fig. 6
Influence of valid EMA steps Teema The valid number of EMA steps, Nemas ensures that the weights of the last nema steps account for more than 90% of the total, while weights beyond this threshold are negligible and can be approximated
invalid. We set nema 100, 1000, and 10000, respectively, to simulate distribution updates at different speeds. We plot the probability curve Pr (c y) during the first five EMA intervals (i.e., E [0, 500) steps for original sampling, with the first half held O because the BNF scheduler has not yet sampled the current interval), The results in Fig. show that an overly long EMA time step prevents the model from capturing the real- inability correct distribution bias. In practice, typically
time pose distribution, leading choose nema 100 to ensure effective estimation of the distribution.
Influence of number of views ng. We study how varying the number of pose categories (n2 3, 4, and 6) affects overall performance. Our experiments, as illustrated in Fig. [ show that increas- ing the number of categories from 4 to 6 produces minimal improvement in 3D consistency. This
25

---

Published as a conference paper at ICLR 2025
"A photo of a beagle's 's head wearing beret."
(a) Sketches
(b) Samples test
(c) Cross modality rectification
(d) Cross modality control
Figure 11: Cross-modaliiy generation results. (a) Amateur sketches drawn by the authors. (b) Initial Stable Diffusion generations for evaluation (quantitative results in Table (c) Results with sketch- based rectification. (d) Generations with pose-controlled sketch guidance, demonstrating significant quality improvements.
performance plateau stems from our pose classifier's design, which lacks sensitivity to fine-grained pose distinctions, creating natural ceiling when presented with more detailed pose categories.
Sampling We examine an alternative approach for sampling the target distribution using es- timated probability (ely)y, similar the first-stage methodology DreamControl (Huang al., 2024). As shown in Fig. this sampling strategy overemphasizes densely populated regions of the pose space while providing insufficient supervision for less frequent viewpoints. This imbalance leads to compromised geometric consistency in the generated results.
C.3 TEMPLATES FOR POSE CLASSIFIER
We generate reference template images using Stable Diffusion (Rombach et 2022 by combining the original prompt with directional text modifiers: "from front view," "from side view," and "from back view." With the of generated images, users can simply select some with different poses as the templates. Additionally, users also have the option to generate templates using Zero-1-to-3 (Liu ct al. 2023).
As shown in Fig. 10 the template images do not need to be 3D consistent. They only need to convey basic pose information. Our experiments Appendix C.44 demonstrate this flexibility through cross-modal experiment where successfully use simple sketches templates.
C.4 CROSS MODALITY RECTIFICATION
We demonstrate our pose estimator's scalability through a cross-domain experiment using hand- drawn sketches. For this experiment, we draw four sketches (see Fig. a)) corresponding to the prompt and construct a pose classifier based on these sketches. When applied to real images, this sketch-based based classifier accurately computes both masks and probabilities. In Fig. [T)b), sample some images for testing the performance of the fretch-based classifier. Quantitative results are presented in Table
26

---

Published as a conference paper at ICLR 2025
Table 2: Cross-modie classifier predictions for images in Fig. 11. Images are arranged in two rows: IDs 1-6 (top) and IDs 7-12 (bottom). Cell colors indicate manually annotated ground-truth pose categories. Note: Ambiguous images (e.g., ID that could be classified as multiple categories (e.g. "front" or "left") are highlighted with lighter color. Bold values indicate maximum classifier probabilities.
ID
Back
Front
Left
Right
ID
Back
Front
Left
Right
0.0127
0.7895
0.1773
0.0204
0.0017
0.4411
0.5447
0.0123
0,0037
0.2843
0.0217
0.6902
0.0100
0.1306
0.0069
0.8522
3
0.0013
0.9684
0.0152
0.0149
0.8361
0.0154
0.0792
0.0691
0.0016
0.8374
0.1291
0.0318
10
0.0006
0.1418
0.8526
0.0048
5
0.0781
0.0637
0.0004
0.8575
11
0.8979
0.0610
0.0236
0.0173
0.0018
0.1153
0.8795
0.0033
12
0.0393
0.0368
0.0055
0.9182
"Samurai koala bear."
"Wes Anderson style Red Panda, reading a book, super cute, highly detailed and colored."
(a) Front
(b) Back
(c) Left
(d) Right
Figure 12: Pose-controlled generation using classifier outputs. Each batch demonstrates generation results using one of the four classifier logits, corresponding to "front", "back", "left", and "right" poses, respectively.
Fig. (T)) and (d) present the 3D generation results. (c) shows scenes generated using manual sketch guidance, while (d) demonstrates results using pose control supervision (detailed in Appendix C.5 The latter approach notably enhances the 3D consistency of the output.
C.5 CONTROLLABILITY AS A SPECIAL CASE OF USD
We further explore conditional control building upon our previous classifier. While our experimental setup considered a uniform distribution of poses across all angles, real-world applications may not require such comprehensive coverage. The number of discrete poses can be reduced when we can specify more precise range of angles.
In cases where the perspective can be constrained narrow range (e.g., front views from -45 to 45 degrees) that can be summarized with only one template image, the rectifier function can be simpli- fied to r (arele (y)) = (c( v). This simplification similar to approaches used in PnP (Graikos ct al. 2022) and DPS (Chung et al. 2022), where irrelevant (cly) terms are eliminated. Under these conditions, the generation process controlled by single pose, effectively targeting just one specific conditional sub-distribution.
For implementation, can optimize the classifier using single logit to achieve this control. While this approach resembles PnP (Graikos et al. 2022). our method enables multi malli--article optimization without requiring specialized elassifiers for noisy images, which turns out be more flexible.
27

---

Published as a conference paper at ICLR 2025
Table Runtime comparisons. All measurements are reported in minutes.
USD 297.34
VSD 180.70
SDS 43.35
Debiased-SDS
PerpNeg 45.97
ESD 201.55
SDS-Bridge 73.24
43.58
Table 4: User study on geometric consistency. Experts ranked the generated results from different baseline methods based on geometric consistency. Rankings are converted scores based posi- tion, with higher-ranked options receiving higher scores.
USD
VSD
SDS
Debiased-SDS
PerpNeg
ESD
SDS-Bridge
4.80
3.27
2.81
2.44
1.96
3.21
2.50
Table 5: Success rates for Janus-free generation. Scoring system assigns penalties of 0.5 for local feature duplication and 1.0 for global feature duplication. While our method effectively mitigates global feature duplication, local duplications occasionally persist due inherent limitations of score distillation algorithm.
Prompts
Mean
Std
Median
Mode
Min
Max
"kangaroo" "bear" 0.94
0.65
0.5296
0.5
0.5
0
1.5
0.5270
0.75
0.5/1
0.5
2
As demonstrated in Fig. 12 our approach achieves precise pose control in 2D particle generation. The examples show more efficient control compared to using textual descriptions. We've 'v'e success- fully extended this functionality to both cross-modal (sketch-guides and 3D-prior based generation, enhancing geometric consistency as shown Fig. 11 d)) and Fig. [6[b). Furthermore, by leveraging can explore novel applica-
DINOv2 (Oquab al. 2023)'s 's cross-modal matching capabilities, tions. For instance, using airplane images to guide the generation of flying eagles. We believe this opens up promising avenues for expanding the practical applications of our theoretical framework.
C.6 RUNTIME EVALUATION
We conduct our experiments at 256 x 256 resolution using a single Nvidia GeForce RTX 4090 GPU. While USD and VSD share the same framework, other comparison methods are implemented using threestudio (Guo al., 2023). To ensure a fair comparison, all methods are evaluated using their default settings to achieve convergence. As shown in Table 3 USD requires longer computation times compared to baseline methods, primarily due to the additional back pack-propagation through the diffusion U-Net (Ronneberger et al. 2015) required by the rectifier function.
Despite the increased computational overhead, our method's substantial improvements geometric consistency justify this trade-off. The computational cost remains practical for pose control applica- tions (in Appendix C.5), as pose control is typically only necessary during the initial stages of shape formation. Future research directions could focus developing optimization strategies to enhance computational efficiency while maintaining the method's 's performance advantages.
C.7 USER STUDY AND DISCUSSIONS ON THE SUCCESS RATE
We conduct a user study involving more than 25 human experts to evaluate and rank different meth- ods based their overall geometric consistency. The average ranks for all methods are reported in Table
Additionally, we analyze the success rate of generating Janus-fiee freus-free models. Given the inherent ran- domness of score distillation methods, achieving perfect geometric consistency remains challeng- ing. To quantify inconsistencies, we develop systematic rating system. The system evaluates both global and local features using the formula Rscore (n ) + 0.5 (n - n where for global features (such as faces or body), n2 ny represents the difference between the actual count and the expected count, while for local features (such as legs, arms, tails), each duplicated feature contributes 0.5 to the score. Here, // and n', denote the actual and expected counts of local features respectively. Note that the expected count is the correct number that the scene should occur. For instance, in the case of generating bust, the face's expected count is 1. Table presents
28

---

Published as a conference paper at ICLR 2025
Table 6: Classification performance and ablation studies. We compare three variants: orientation classifier without texture score, texture classifier without orientation score, and the complete model using both scores.
Metric
Full
Orient
Texture
Average Accuracy
0.7846
0.6328
0.7699
Average Precision
0.7986
0.6718
0.8013
Average Recall
0.7426
0.5934
0.7396
Average Score
0.7439
0.5942
0.7308
"Samurai koala bear."
"A kangaroo wearing boxing gloves."
(a) VSD
(b) USD
(c) VSD
(d) USD
Figure 13: 2D score distillation comparing VSD (Wang et al. 2024b) and USD. The prompts are augmented with auxiliary view descriptions ("from side view, from back view") to capture multi- perspective information. Due to the original distribution's 's bias toward back-view angles, VSD gen- erates predominantly back-vie view results, while USD successfully rectifies this distributional bias to produce more balanced viewpoints.
the Multi-Face Janus scores for two test prompts, demonstrating our method's performance. We provide detailed discussion of potential solutions in Appendix
D CLASSIFIER EXPERIMENTS
In this appendix, we evaluate our pose classifier's performance using the annotations described in Sec. 4.1 We assess both the texture and orientation branches independently. Table 6 presents the classification performance of the main classifier and its two variants. This ablation study validates our chosen classifier architecture.
E VALIDATION EXPERIMENTS ON 2D SAMPLING
We validate the performance of uniform score distillation using set of 2D particles. Starting with 16 initialized particles, we conduct training over 4,000 iterations. The comparison between USD and variational score distillation is illustrated in Fig. 13 Our results demonstrate that USD successfully achieves a more balanced distribution compared the biased pre-trained distribution.
Fig. 14 shows the pose distribution statistics across 10 intervals for both USD and VSD. Here, (.(yy) which represents the expectation of (:la,, y) over æt, indicates the current pose distri- bution and reveals training bias progression. While VSD exhibits strong bias toward specific distributions during training (due to the usage of auxiliary prompts), our method maintains an ap- proximately uniform distribution throughout the process.
DISCUSSION ON LIMITATIONS AND FUTURE WORKS
This discussion examines our work's 's boundaries while identifying promising paths for subsequent research. We identify several key limitations and opportunities for advancement.
A primary limitation of this work is generation speed. The bottleneck lies the U-Net (Ronneberger al. 2015) gradient poopppropagttion introduced by the rectifier function, which requires further
29

---

Published as a conference paper at ICLR 2025
(r(Cy) y) for VSD
Hign
200
ters
1ters
180
2:00
-50
2012
100
Pe ( for USD
-4000
4400
Rofs
Itors
Iters
0.2
10500
10er5
$10
Riers
iteer
Figure 14: Comparison of pose probability distributions p. (Glv) between VSD WWag et al. and USD across different timestep intervals While VSD converges to a biased distribution, USD maintains an approximately uniform distribution across camera poses.
2024b)
"A person's 's face."
Original (VSD)
(b) Rectified (USD)
Control (male)
(d) Contorl (female)
Figure 15: Addressing semantic distributional bias using a CLIP (Radford et al. 2021 classifier.
(a) Results from VSD (Wang al, 20246) exhibit inherent gender bias, predominantly generating female subjects. (b) By incorporating CLIP-basee male/female classifier, our method achieves balanced gender distribution. (c) and (d) demonstrate fine-grained control over specific gender at- tributes, enabling targeted generation of male and female subjects respectively.
optimization. Future research could explore methods to effectively bypass U-Net gradient back- propagation or develop a score-free optimization framework similar to MieroDreamer (Chen et al.) 2024).
Another significant challenge concerns 3D consistency of localized features. While USD eliminates bias in the overall data distribution, its reliance on the score distillation algorithm, which lacks ex- plicit geometric consistency supervision, can lead to geometrically inconsistent content, potentially limiting practical applications. Addressing this limitation requires incorporating multi pusti-perspective supervision during generation. Notably, the special case discussed in Appendix C.5 demonstrates a potential supervision mechanism for score distillation that warrants further investigation.
Beyond current technical limitations, we propose new directions for control-based synthesis that expand cross-modal approaches (see Appendix C.4). For instance, Fig. 16 demonstrates
30

---

Published as a conference paper at ICLR 2025
"A platypus, dressed in video game pixelated costume, steps pixelated surfboard and holds squid weapon that emits 8-bit light effects."
(a) Tripo Al v2
(b) USD+control
Figure 16: Demonstration of the integration of USD with 3D-based methods (Tripo AI).
periment using an imaginative prompt. As shown in Fig. T6 a), the 3D generative model Tripo Al captures basic geometric elements effectively, but still faces challenges when interpreting more abstract or imaginative descriptions (i.e., "pixelated costume" and "pixelated surfboard") due to its 3D modeling constraints. In contrast, control (Fig. [)),)) resulting in a more accurate prancing effect that better matches the text descrip- tion, demonstrated in Fig. [6].). While our model trained from scratch and may lack geometric refinement, fine-tuning it from a geometrically consistent base model (Zheng et al. 2024) can yield results that excel in both geometric accuracy and textural detail.
approach leverages selected Tripo Al renderings for pose
Finally, highlighting the versatility of our approach, our USD algorithm demonstrates considerable extensibility beyond pose classification and 3D generation. Fig. 15 showcases its application in ad- dressing gender distribution bias in image generation using the CLIP (Radford et al. (021) classifier, enabling independent control over gender representation. This adaptability suggests that USD could be applied to address other forms of algorithmic bias with different classifier architectures.
G PROMPTS
In our experiments, we use the prompts introduced in the existing works such as SDS (Poole et al., 2022 and VSD (Wang et al. 2024b) as shown Table 7
Table Experimental prompt list. Each prompt is augmented with auxiliary view descriptors "from side view, from back view".
ID
Prompt Description
ID
Prompt Description
An airplane made out wood.
12
A peacock on a surfboard.
2
A bald eagle carved out of wood fea-
13
A portrait of Groot, head, HDR, photo- realistic, 8K.
tures.
A blue motoreycle.
14
sea turtle.
4
A dragon-shaped teapot.
15
A zombie bust.
5
A DSLR photo of a beagle in detec-
I6
A 3D printed white bust of a man with curly hair.
tive's outfit.
6
A DSLR photo of chimpanzee
17
DSLR Camera, photography, dsir, camera, noobie, box-modeling, maya.
dressed like Napolcon Bonaparte.
7
A DSLLR photo of football helmet.
18
Mecha vampire girl chibi.
8
A kingfisher bird.
19
Robot with pumpkin head.
A fantasy painting dragoncat.
20
Robotic bee, high detail.
10
A kangaroo wearing boxing gloves.
21
Samurai koala bear.
11
A DSLR photo of a squirrel playing
22
Wes Anderson style Red Panda, read- ing book, super cute, highly detailed and colored.
guitar.
https://ummllass.iigeni
31

---

Published as a conference paper at ICLR 2025
H ADDITIONAL COMPARISON AND RESULTS
In this part, we present additional comparisons and results.
DSLR photo of beagle detective's outfit."
ESD
SDS bridge
USD
"A portrait of Groot, head, HDR, photorealistic, 8K.
ESD
SDS bridge
USD
"A kangaroo wearing boxing gloves."
ESD
SDS SDS-bridh bridge
USD
Figure 17: Additional Comparisons with ESD and SDS-Bridge.
32

---

Published as a conference paper at ICLR 2025
"Samurai koala bear."
SDS
Debiased
PerpNeg
VSD
USD
"DSLR Camera, photography, dslr, camera, noobie, box-modeling, maya."
SDS
Dehiased-S SDS
PerpNeg
VSD
USD
Figure 18: Additional Comparisons.
33

---

Published as a conference paper at ICLR 2025
"A portrait of Groot, head, HDR, photorealistic, 8K."
SDS
Debiased-SDS
PerpNeg
VSD
USD
"A DSLR photo of football helmet."
SDS
Debiased-Si SDS
PerpNeg
VSD
USD
Figure 19: Additional Comparisons.
34

---

Published as a conference paper at ICLR 2025
"A kangaroo wearing boxing gloves."
SDS
Debiased-SDS
PerpNeg
VSD
USD
"A DSLR photo squirrel playing guitar."
SDS
Rebiased-SDS
PerpNeg
VSD
USD
Figure 20: Additional Comparisons.
35

---

Published as a conference paper at ICLR 2025
"A DSLR photo of beagle in detective's outfit."
"Mecha vampire girl chibi."
"A peacock on surfboard." T*
"A blue motorcycle." "
"An airplane made out of wood." "
"Robotic bee, high detail." "
"A DSLR photo of a chimpanzee dressed like Napoleon Bonaparte."
Figure 21: More examples.
36

---

Published as a conference paper at ICLR 2025
"A bald eagle carved out of wood features."
"A kingfisher bird."
"Robot with pumpkin head."
"A dragon-shaped teapot."
"A sea turtle."
"A zombie bust."
"A fantasy painting of dragoncat."
Figure 22: More examples.
37


Reviews:
[{"review_id": "5glYm4NiPN", "reviewer": "Reviewer_i678", "summary": "The authors of the paper introduce RecDreamer, a text-to-3D generation task that aims to reshape the underlying data distribution of pretrained text-to-image diffusion models to eliminate the Multi-face Janus problem. To achieve this, they develop an auxiliary function derived from the joint distribution expression of the data x and camera pose c. A key component of this auxiliary function is a well-designed, lightweight pose classifier capable of calculating p_t(c|x_0). Extensive experiments demonstrate the effectiveness of RecDreamer in eliminating the Janus problem.", "questions": "I would appreciate it if the authors could provide a clear explanation of the second and third weaknesses. I am open to revising my score if the author addresses this concern.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1)\tThe authors present an insightful analysis of the emergence of the multi-face Janus problem in text-to-3D generation. Building on their analysis, they offer a solid theoretical framework on reshaping the underlying data distribution of pretrained text-to-image diffusion models.\n2)\tThe design of the pose classifier appears to be innovative, with clear and comprehensive details provided in the appendix.\n3)\tThe qualitative and quantitative results of RecDreamer demonstrate its state-of-the-art performance, as validated in their experiment section.", "weaknesses": "1)\tThe evaluation of the paper is limited. Initially, all experiments are only tested on 22 prompts from the original DreamFusion gallery, while previous works typically use over 40. The prompt list is also not provided in the appendix, which compromises the reproducibility of the experiments. Furthermore, the absence of recent open-sourced baseline methods that focus on addressing the Janus problem, such as ESD1 and JointDreamer2, is notable.\n2)\tThe authors introduce the concept of the \"joint distribution expression of the data and camera pose,\" but provide limited explanation of the practical meaning of such a distribution. Also, the use of a rather discrete 4-views to represent the camera pose distribution does not appear to be meaningful. More ablation studies should be conducted.\n3)\tThe authors state that the image templates are \"user-provided\" in line 317 of the main manuscript, which seems unusual. It is not feasible for users to provide consistent multi-view templates for any text prompts.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jahMTec3Zj", "reviewer": "Reviewer_wW4M", "summary": "This paper tackles the Janus problem with score distillation-based text-to-3D generation via debiasing the camera pose distribution of the pre-trained text-to-image diffusion model. Specifically, the authors derive an auxiliary function that rectifies the pose distribution induced by the original image distribution to a targeted distribution (e.g. uniform). Consequently, the score distillation rule yielded from the debiased image distribution, termed uniform score distillation (USD), is derived as a combination of the variational score distillation plus the “score” of the auxiliary function. The auxiliary rectifier can be computed with a pose classifier and a running estimator of the pose distribution over the Gaussian perturbed image distribution.", "questions": "1. Why is the VSD loss in Eq. 14 denoted as $L’_{VSD}$ instead of $L_{VSD}$​ as in Eq. 9?\n\n2. Additionally, I wonder whether modifying the camera sampling distribution in VSD to use the estimated $p(c∣y)$ could yield similar or complementary effects in mitigating Janus problems.", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "+ This paper offers a distinctive approach to addressing the Janus problem in score distillation-based text-to-3D generation. Its central argument asserts that the pre-trained image distribution is intrinsically biased toward canonical views, causing the resulting 3D content to exhibit this same bias—manifesting as the Janus problem in practical applications. This perspective contrasts with most existing score distillation methods, which often assume that the 2D pre-trained distribution is ideal and strive to align the view distribution with it more closely.\n+ The proposed method is both novel and theoretically grounded, offering a unified and principled approach to debiasing that integrates seamlessly into current score distillation frameworks. The debiasing mechanism is based on solid theoretical underpinnings yet is surprisingly straightforward to implement. All derivations appear correct. The proposed estimation for the otherwise intractable auxiliary rectifier term is thoughtfully designed, with each step justified through theoretical insight.\n+ The paper is dense and well-written, with thorough engineering details, including but not limited to an efficient pose classifier, a running estimator of the pose distribution, and improved training techniques. The results are extensive: beyond fundamental qualitative outcomes, the paper includes numerous validation experiments in the appendix, demonstrating the efficacy of each design element.", "weaknesses": "- The quantitative results in this paper are somewhat limited. While I recognize the lack of a standard metric for evaluating generated 3D results, a few commonly used measures, such as the CLIP score demonstrated in SDS [1], could still be applied. Given the high degree of randomness often seen in score-distilled outputs, it would also be beneficial to assess the method using human evaluation, specifically by measuring the success rate of Janus-free results [2].\n\n- Another limitation is that the proposed USD appears to require reference images from various view angles, which may be impractical when the prompt is abstract or imaginary and lacks similar images on the shelf. The authors may wish to discuss potential solutions to this constraint.\n\n- The paper also overlooks several related works in the text-to-3D generation literature. While the field is vast, I suggest including a more comprehensive review. Some key omissions in the current version include: [3,4,5,6]\n\n- Many state-of-the-art approaches currently rely on training or fine-tuning generative models on 3D data. It would be interesting to show whether score distillation with the rectified distribution could largely match the performance of the distribution fine-tuned with extra data. I’d also suggest demonstrate whether the proposed USD can even enhance these training-based methods.\n\n\n[1] Poole et al., DreamFusion: Text-to-3D using 2D Diffusion\n\n[2] Wang et al., Taming Mode Collapse in Score Distillation for Text-to-3D Generation\n\n[3] Yu et al., Text-to-3D with Classifier Score Distillation\n\n[4] Katzir et al., Noise-free Score Distillation\n\n[5] Wang et al., SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via Stein Identity\n\n[6] Shi et al., MVDream: Multi-view Diffusion for 3D Generation", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "02oro8Q6xM", "reviewer": "Reviewer_1nXi", "summary": "This paper proposes to rectify the biased pose distribution, ensuring the pose variation is uniformly distributed, to solve the Janus problem. A uniform score distillation module is introduced accordingly.", "questions": "Can the authors report the runtime per sample of your method and other baselines to ensure the completeness of comparison?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper rectify the biased distribution to uniform distribution by reweighing the density of original distribution.\n2. A corresponding uniform score distillation process is designed to improve the consistency.", "weaknesses": "1. More comparison with baselines is needed, such as [1].\n2. Runtime analysis is missing, the sampling time per image is expected.\n\n\n[1] McAllister, David, et al. \"Rethinking Score Distillation as a Bridge Between Image Distributions.\" arXiv preprint arXiv:2406.09417 (2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "H4kxp8slvv", "reviewer": "Reviewer_X3Gz", "summary": "The paper presents RecDreamer, an approach to address the Multi-Face Janus problem in text-to-3D generation, which arises from geometric inconsistencies across different poses of 3D assets. The authors propose a technique called uniform score distillation, which modifies the underlying data distribution to ensure pose variation follows a uniform distribution. This is achieved by a rectification process that adjusts the density distribution, facilitated by a training-free classifier that estimates pose categories. The approach aims to eliminate biases towards a canonical pose and improve geometric consistency without compromising rendering quality. The experimental results demonstrate the effectiveness of RecDreamer in achieving consistent 3D asset generation across various poses.", "questions": "* How does RecDreamer handle variations in complex textures or intricate details that might not be directly related to pose?\n* Can the proposed method be extended or adapted to address other biases in text-to-3D generation beyond pose inconsistency?\n* What are the computational requirements for implementing RecDreamer, and how does it perform in terms of efficiency compared to baseline methods?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "* An approach to address data bias issue: The paper introduces a solution to a well-known problem in text-to-3D generation by addressing the biases in data distribution through uniform score distillation.\nThe use of a training-free classifier to estimate pose categories is an efficient and novel aspect of the approach, avoiding the need for additional training.\n\n* Comprehensive Methodology:\nThe integration of reverse Kullback-Leibler divergence in the score distillation framework is well-articulated, allowing for the seamless incorporation of the rectified distribution.\nThe method's ability to maintain rendering quality while resolving the Multi-Face Janus problem is a significant advantage.\n\n* Experimental Validation:\nThe paper provides thorough experimental validation, demonstrating the method's effectiveness in improving geometric consistency across different poses.\nAdditional experiments on 2D images and a toy dataset further substantiate the robustness of the algorithm.\n\n* Broader Applicability:\nThe potential applications of the pose classifier beyond the primary task highlight the versatility and potential impact of the proposed approach.", "weaknesses": "* Complexity and Scalability:\n The introduction of an auxiliary function and classifier adds complexity to the method. Details on computational efficiency and scalability in large-scale applications are lacking.\n\n* Generalization:\nWhile the method is effective for the specific problem addressed, there is limited discussion on its generalizability to other types of biases or different domains within 3D generation.\n\n* Quantitative Metrics:\nThe paper would benefit from a more detailed presentation of quantitative metrics used to evaluate geometric consistency and rendering quality, alongside comparative analysis with existing methods.\n\n* Background section is too long. This paper put many equations in Sec.2, but it actually did not provide strong support to the proposed uniform score distillation. Some space could be saved to explain Sec. 3.3.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)