StochSync: Stochastic Diffusion Synchronization for Image Generation in Arbitrary Spaces
Authors: Kyeongmin Yeo
Jaihoon Kim
Minhyuk Sung
Venue: ICLR 2025 Poster (2025)
Keywords: Diffusion Models
Synchronization
Score Distillation
Panorama
Texturing
Primary area: generative models
TL;DR: We propose a zero-shot state-of-the-art method for generating images in arbitrary spaces (e.g., a sphere for 360-degree panoramas and a mesh surface for texture) using a pretrained image diffusion model.

Abstract: We propose a zero-shot method for generating images in arbitrary spaces (e.g., a sphere for 360◦ panoramas and a mesh surface for texture) using a pretrained image diffusion model. The zero-shot generation of various visual content using a pretrained image diffusion model has been explored mainly in two directions. First, Diffusion Synchronization–performing reverse diffusion processes jointly across different projected spaces while synchronizing them in the target space–generates high-quality outputs when enough conditioning is provided, but it struggles in its absence. Second, Score Distillation Sampling–gradually updating the target space data through gradient descent–results in better coherence but often lacks detail. In this paper, we reveal for the first time the interconnection between these two methods while highlighting their differences. To this end, we propose StochSync, a novel approach that combines the strengths of both, enabling effective performance with weak conditioning. Our experiments demonstrate that StochSync provides the best performance in 360◦ panorama generation (where image conditioning is not given), outperforming previous finetuning-based methods, and also delivers comparable results in 3D mesh texturing (where depth conditioning is provided) with previous methods.

Published as a conference paper at ICLR 2025
STOCHSYNC: STOCHASTIC DIFFUSION SYNCHRONIZA- TION FOR IMAGE GENERATION IN ARBITRARY SPACES
Kyeongmin Yeo" Jaihoon Kim
Minhyuk Sung
KAIST laaaaa, jh2 7kim,
mhsung @kaist............k. Sakiist ac kr
81
Figure 1: Assorted mesh textures and panoramas generated using StochSyne, including one the background (environment map), which is 360 panorama. StochSync extends the capabilities of image diffusion models trained in square spaces to produce images in arbitrary spaces such as cylinders, spheres, tori, and mesh surfaces.
ABSTRACT
We propose zero-shot method for generating images in arbitrary spaces (e.g., sphere for 360 panoramas and mesh surface for texture) using a pretrained image diffusion model. The zero-shot generation of various visual content using a pretrained image diffusion model has been explored mainly in two directions. First, Diffusion Synchronization-pe perfoormng reverse diffusion processes jointly across different projected spaces while synchronizing them in the target generaass high-quality outputs when enough conditioning is provided, but it struggles in its absence. Second, Score Distillation Sampling-graduall updating the target space data through gradient descent-results in better coherence but often lacks detail. In this paper, we reveal for the first time the interconnection between these two methods while highlighting their differences. To this end, propose StochSyne, a novel approach that combines the strengths of both, enabling effective performance with weak conditioning. Our experiments demonstrate that StochSync provides the best performance in 360 panorama generation (where image conditioning is not given), outperforming previous finetuning- methods, and also delivers comparable results in 3D mesh texturing (where depth conditioning is provided) with previous methods. Project page is at https: 7/stochsync. github..o//
INTRODUCTION
Diffusion models pretrained on billions of images (Rombach et al. 2022 Midjourney have demon-
strated remarkable capabilities various zero-shot applications. A notable example is the zero-shot generation of diverse visual data, including arbitrary- images (Bar- Tal et al. 2023 Lee et al.) 2023), 3D mesh textures (Cao et al. 2023), ambiguous images (Geng et al. 2024b), and zoomed-in
"quual contribution

---

Published as a conference paper at ICLR 2025
images (Wang et al., 2024a Geng et al., 2024a). This extension to other types of data is achieved through mapping from the space in which the diffusion models are trained (referred to as the instance space) to the space where the new data is generated (the canonical space). For instance, while a 21) square is the instance space for typical image diffusion models, a cylinder canonical space for generating 360 panoramic images, and a 3D mesh surface becomes the canonical space for mesh texture generation. Examples are shown in Fig. T Such zero-shot generation in the canonical space allows for the effective production of various types of data without the need for new data collection training a separate generative model for each data type.
sphere serves as the
There have been two main approaches to addressing this problem. The first is Diffusion Synchroniza- tion (DS) (Bar-Tal et al., 2023] Kim al. 2024a), which performs the reverse generative process of diffusion models jointly across multiple instance spaces while synchronizing intermediate outputs by mapping them to the canonical space. This approach has been successfully applied to generating various types of data, though it has a notable limitation: synchronization often fails to converge when strong conditioning, such as depth images, is not provided. As result, the generated outputs frequently exhibit visible seams and fail to smoothly combine multiple projections from the instance spaces. This becomes a critical drawback for certain applications, such as 360 panoramic images, where image conditioning may not be available.
The other line of work is Score Distillation Sampling (SDS) (Poole et al.] 2023) and its vari- ants (Lukoianov et al. 2024, Liang et al.) (024). Unlike DS, SDS does not perform the reverse diffusion process but instead uses pradient-descent-bee canonical space. SDS has been widely applied to the generation of different types of visual data and, compared to DS, has shown greater robustness in scenarios where no image conditioning is provided. However, its quality is less realistic, the generation process is not based the reverse diffusion process, which diffusion models are specifically designed for.
updates from various instance spaces to the
In this work, we introduce novel method named Stochastic Diffusion Synchronization, Stoch for short, which combines the best features of the two aforementioned approaches to achieve superior performance in unconditional canonical data generation. StochSyne is based on our key insights from analysis on the similarities and differences between DS and SDS. Specifically, we observe that each step of SDS can be interpreted as a one-step refinement in DDIM (Song et al. 2021a) while maximizing stochasticity in the denoising. We incorporate this maximum stochasticity into DS, resulting in better coherence across instance spaces and improved convergence. To enhance the realism as well, we propose replacing the prediction of the clean sample at each denoising step from Tweedic's s formula with a multi-step denoising process, and also using non oon-overlap views for the instance space while achieving synchronization over time through the overlap of views across different time steps. Notably, from the SDS perspective, St ochhSnnc can also be seen as modifying SDS by changing the random time sampling to a decreasing time schedule, resembling the reverse process, and by replacing the gradient descent with fully minimizing the 12 loss.
In the experiments, test StochSyne on two applications: 360 panoramic image generation and mesh texture generation. The former represents the unconditional case (except for a text prompt), while the latter is the conditional case with a depth map as the input. For the panoramic image generation, we demonstrate state-of-the-art performance compared to previous zero-shot (Cai et al., 2024) and finetuning- method does not suffer from overfitting issues, unlike methods finetuned on small-scale panorama datasets (Chang al., (017). and it avoids geometric distortions that occur with inpainting- based methods (Cai et al. a 2024). For mesh texture generation, although our method is designed to focus on the unconditional case, demonstrates comparable results to previous DS methods (Kim et al.
methods (Tang et al., 2023b Zhang et al. 2024a). Notably, our zero-shot
2024a) and outperforms other prior works (Youwang et al. 2023 Zeng et al. 2024 2023a Richardson et al. 2023).
Chen et al.
2 RELATED WORK
In this section, we first review approaches that generate samples in canonical space by leveraging pretrained diffusion models trained in instance space: Diffussio Synchronization and Score Distilla- tion Sampling. We then discuss these approaches, along with other related works, in the context of two applications: panorama generation and 3D mesh texturing.
Diffusion Synchronization (DS). Liu al. (022 was among the first works to utilize DS, focusing on compositional image generation. Subsequent works, such as (Bar- et al., [023] Lee et al. 2023). extended DS to support image generation at arbitrary resolutions. Beyond images, DSS has been
2

---

Published as a conference paper at ICLR 2025
widely applied to generate textures for meshes (Liu et al. 2023 Zhang et al. 2024b Chen et al. 2024a), long animations (Shafir et al. 2024), and visual spectrograms (Chen et al. (024b). Recently, Kim et al. 2024a) provided an in-depth analysis of previous DS-based methods and introduced method demonstrating superior performance across diverse applications, which we will use as the base DS method. While DS performs well under strong input conditions (e.g.,depth images), struggles to generate plausible data points when the input conditions weak.
Score Distillation Sampling (SDS). DreamFusion (Poole et al. 2023) first introduced SDS to generate 3D objects from text prompts, and several subsequent works have aimed to improve its quality (Wang et al.] 2024b; Katzir et al., [203)] /Zhu et al., 2023) and running time (Huang et al.
2023; Tang et al. 2023ap. ISM (Liang et al. 2024) and SDI (Lukoianov et al. 2024) utilized DDIM inversion to obtain noisy data points. Beyond 3D generation, SDS has been widely applied in various fields, including image editing (Hertz et al. 2023), 3D scene editing (Koo et al. 2024 Park et al. 2023), and mesh deformation (Yoo et al. 2024). However,
methods often produce
suboptimal samples lacking fine details compared to reverse process outputs. We also discuss the differences between our method and recent variants of SDS in Sec. 6
Panorama Generation. In text-conditioned panorama generation, Tex12Light (Chen et al. 2022 employed VQGAN (Esser et al., 202I) with multi-stage pipeline. With the release of image diftusion models trained on large-scale datasets (Rombach et al. 2022). approaches leveraging pretrained diffusion models have gained attention. MVDiffusion (Tang et al. (023b) and PanFusion Zhang et al. 2024a) finetune these pretrained models using a panoramic images dataset (Chang al. 2017). However, finetuning diffusion models on a small dataset risks overfitting, reducing their generalizability. In contrast, SyneTweedies (Kim al.) 2024a) employs DS for zero-shot panorama generation but relies on depth map conditions, which are not commonly available in practice. L- MAGIC (Cai et al. 2024). on the other hand, adopts an inpainting diffusion model, sequentially filling in the panoramic images. However, this iterative process cannot refine previous predictions, leading to error accumulation and often resulting wavy panoramas.
Mesh Texturing. 3D mesh texturing using image diffusion models has gained significant attention. Among these approaches, Paint3D (Zeng et al. 2024) finetunes pretrained diffusion model on synthetic 3D mesh dataset (Deitke al. (023), but this often results unrealistic texture images due to overfitting to the synthetic dataset. For zero-shot approaches, previous works have utilized SDS to update the texture of 3D meshes (Metzer et al., 2023 Chen et al. 2023b Youwang et al. 2023) DS is also widely used for 3D mesh texturing, with previous works (Liu et al. 2023: ZZhang et al.
2024b Kim et al. 2024a) averaging the one-step predicted clean samples across multiple denoising processes. Another line of research explores the outpainting approach (Chen et al. 2023a Richardson
ct al. 2023). where the 3D mesh is textured iteratively, often resulting in textures with visible seams.
"Majestically rising towards the heavens, the snow-capped mountain stood."
(a) SyncTweedies Kim al. 2024a
(b) SDS (Poole et al.
2023
(c) Sync' Sweedeee + Max d:
(d) SyncTweedies + Max ar Impr.Xoje
(e) SDI [Lukoianev
2024
(I) StochSyne
Figure 2: A comparison of SyncTweedies (Kim al. 2024 a synchronization method, SDS (Poole et al., (023), and StochSync which uses SyncTweedies as base and incorporates maximum stochasticity (Max ot), multi-step Xy computation (Impr. Xujc), and non-overlapping view sampling (N.O. Views), alongside others that use only a subset of these components.
3 PROBLEM DEFINITION AND OVERVIEW
We propose a method for generating data points in one space (referred to as the canonical space 2) using a pretrained diffusion model that has been trained another space (referred to as the
3

---

Published as a conference paper at ICLR 2025
instance space X), where the mapping from the canonical space to the instance space is known. For example, the canonical space could be a sphere representing 360" panoramas, for creating mesh textures, and the instance space is a 2D square, the space for most pretrained image diffusion models. In general, a region of the canonical space is mapped to the instance space through a specific view. The mapping from a region of the canonical space to the instance space through view is represented by the projection operation fe(z) Ze X, where E Ze C Z. Our objective is to produce realistic data points in the canonical space without using any generative model trained on samples in that space, but by leveraging pretrained diffusion models in the instance spaces and their multiple denoising processes from different views. This approach can extend the capabilities of pretrained diffusion models to produce diverse types of data, eliminating the need to collect large-scale data and train separate generative models.
3D mesh surface
In the following sections, we first review the reverse process of a diffusion model (Sec. 4 and two approaches, Diffusion Synchronization (DS) and Score Distillation Sampling (SDS), which generate data points in the canonical space by leveraging pretrained diffusion models instance spaces (Sec. 5). Based on our analysis of the connections and differences between these methods, we propose a novel approach that combines the best features of both and provides an interpretation of the method from the perspectives of DS and SDS (Sec. 6
DIFFUSION REVERSE PROCESS
The forward process of a diffusion model (Sohl-Dicksteins et al. (2015); Ho et al. (2020); Song et al. (2021b)) sequentially corrupts sample data using predefined schedule a...... OT where one can sample x, at arbitrary timestep from a clean sample Xe:
X Vaixo + vI agc, where c~N(0.,1.
Song et al. (2021a) propose DDIM, a diffusion reverse process generalizing DDPM (Ho et al. 2020)
by defining the posterior distribution 9a. (x+- |x, xo) with parameter d, determining the level of stochasticity as follows:
(Xo, fa. (x,- (X,, xo) N (Xo, o;1)
(2)
where Po. (Xo, Xx) = var-ix0 V a1-1 of X, VarXo
ag
In the reverse process, Po (X--1 (x.) becomes the same with the distribution in Eq. while the clean sample Xg approximated using the noise predictor co y), where y is the input condition (e.g.. text prompt); note that the time input is omitted for simplicity. We denote e, cx y), then the prediction of clean sample Xg at timestep 4, Xujt. derived as follows based on Tweedie's formula (Robbins 1956)):
Xuie (x Ex) X2 VI- -,EE
(4)
V Vat.
clean data sample Xy is then generated by first sampling standard Gaussian noise Xy ~ (0, and gradually denoising it over time by iteratively sampling x from pe (x-1//x)) The mapping from a noisy data point x, to X₂ becomes deterministic when a, = 0 for all and equivalent to solving an ODE (Song al. 2021b Chen et al. 2018) with a specific discretization.
Reverse Process from the Perspective of Xnj:- Here, connect the reverse process of DDIM to the algorithms to be introduced in the next section, we reinterpret the reverse denoising process as an iterative refinement process of the prediction of clean sample Xojr- See Alg. [ where Xoje and e are computed at each timestep. Note that the mean of the distribution pa (x, (x.) in Eq. can be rewritten in terms of X9 and EG:
Na. (Xo, Ei) = 1o--1x0 + / a;-1 - o7 e
(5)
Apart from setting a; 0, one consider special case when o, VI- az-1, which maximizes the level of stochasticity during the sampling process. This cancels out the noise prediction term E, in Eq. 5 We denote this case by overriding noo (:.)) with (-), which now takes single parameter Xo:
pr (xo) var-1x00
(6)
4

---

Published as a conference paper at ICLR 2025
Algorithm 1: Diffusion Reverse Process
Algorithm Score Distillation Sampling (SDS)
Inputs: Toput text prompt
Inputs: A canonical space sample
Outputs: Xo: An instance space sample aligned with
y: Inpui lext prompt
Function Keverse Process (p):
Outputs: x: Canonical space sample aligned with y
Xx ~ (0.
Function SIDS (z, y):
while a not converged do
4- (x(xr.M
B ~ U(U
Samp-eRandowVLeel
Kojr e(x
Xaie +- fc(x)
s
for 1 ....
# Noise prediction not used and thus omitted.
1 N (ioo (Moje: (.)
Fq
X--1 N (Xoje] o21)
Eq
e--1 + en (1
Xoit- (1
Soji 6(x1-1,
Eq
(x) Nojt-1
end
end
A Algorithm 2: Diffusion Synchronization (DS)
Algorithm 4: StochSync
Inputs: y: Input text prompi; e LIN; views.
Inputs:
y: Input text prompt
Outputs: z: Canonical space sample aligned with
Outputs: Canonical space sample aligned with
Function r:S (9t; y, cl-M
Function Schhänn (z, y)
2
x7
()
2.. + Samplelkondverrllaaaagiiiiee
I... N do for
xT (0) for 1... N do
(x 5)
(0 ofT
x
Fqq
(c) g(x))))
end
end
min Σ (1) x
2
arg min > (c( (2) x01z
for T Tip
#e L lixed for all
CloN Samp hwwpp//ooooeeerpppppiy...eee.l..
for I... N do
I....
10
Ais Iocs
10
Dit
fat0 (x)
11
x
(1) N NNoo (Hoje (0 ),20)
Bq
// Noise prediction is not used and thus omitted.
12
=--
+- =0(x(_,
(x (...0
13
Nai
(i)
t vix(a)
Eq
end
arg
Suje-
14
min
end
15
end
5 DIFFUSION SYNCHRONIZATION AND SCORE DISTILLATION SAMPLING
As methods leveraging pretrained diffusion models generate data in other spaces, there have been mainly two approaches: Diffusion Synchronization (DS) (Liu et al. 2022 Geng et al. 2024b Kim al., 2024a) and Score Distillation Sampling (SDS) (Poole et al.) 2023 Wang al. 2024b
Lukoianov et al. 2024 Liang et al. 2024). In this section, we briefly review these methods, analyze the connections between them well as their differences, and discuss the limitations of each method.
5.1 DIFFUSION SYNCHRONIZATION
The idea of Diffusion Synchronization (DS) Liu et al. 2022 Geng et al. 2024b; Kim et al. 2024a) to perform the reverse process jointly across multiple instance spaces while synchronizing the processes through mapping to the canonical space. Among the various options for synchronization, Kim et (2024a) have demonstrated that averaging the predictions of the clean samples Xoje in the canonical space and then projecting back to each instance space provides the best performance across a broad range of applications. Alg. 2 shows the pseudocode, which, at each step, performs one-step denoising of DDIM for each view (lines 11-13), updates the data point in the canonical space z while averaging Xpjt by solving a 12-minimization (line 15), and then projects z back to each space (line 10). The differences from the reverse process of DDIM (Alg. [ are highlighted in blue.
For the stochasticity of the denoising process. typically deterministic DDIM reverse process (a, 0) (Bar-Tal et al. 2023 Zhang al. 2024b) DDPM reverse process (a,
(11 a. 1/11 a,) /1 a,/a, (Liu al. 2023) have been used.
Previous works have shown the effectiveness the synchronization approach generating various types of visual data using pretrained image diffusion models, including depth-conditioned panoramic
5

---

Published as a conference paper at ICLR 2025
images, textures of 3D meshes and Gaussians (Kim et al., 2024a Liu et al., 2023). However, we have observed that this approach requires strong conditioning for each instance such as depth images-to achieve optimal quality.
cases where the input condition not provided, such as generating depth-frre 360" panoramas, the outputs tend to show seams as shown in Fig. 2a), mainly due to the wider data distribution and thus difficulties in achieving convergence during synchronization.
5.2
SCORE DISTILLATION SAMPLING
Score Distillation Sampling (SDS) (Poole et al. 2023) and its variants (Wang et al. 2024b Lukoianoy
al. 2024; Liang al. 2024) are alternatives for generating samples in different spaces. Unlike DS, SDS does not use the reverse diffusion process but instead employs gradient-descent-on updates. The motivation behind SDS to leverage the loss function from noise predictor training to discriminate real data points while projecting the canonical data point fc(z), comuptiin it through the forward process, and then predicting the added noise from it.
To clarify the similarities and differences between SDS and DS, we provide a different perspective on understanding SDS, as shown Alg. aligning each computation with those in DS (Alg. 2 There are several key differences, highlighted as green in Alg. [3] First, the timestep ł is not decreased from T to but is randomly sampled until convergence (line 3). Second, while synchronization approaches typically make the reverse process deterministic (Bar-Tal al. 2023 Zhang et al. (024bb identical to DDPM (Liu et al. 2023). J. SDS uses maximun stochasticity (o. = =V 0t-1), thus eliminating the need to maintain the noise Et. Third, the prediction of the clean sample is updated to the canonical space not by solving the 12 minimization but by performing a single gradient descent step (line 7). SDS was originally introduced to perform gradient descent for the loss ||e co 60(X,-1, 1, y) (while omitting the gradient of the U-Net), where is the standard normal sample used in x sampling, i.e. x μ² (Xojt) + T&E (line 5), while is equivalent to the loss used in DS, fl Xoje- up to scale as explained in Appendix (Sec. A)
As observed in previous works (Kim et al. 2024a; Huo et al., 2024). when input conditions are provided, the quality of SDS DoS-generteed outputs is inferior to that of DS- based methods. However, SDS performs better than DS when no conditions are given (except for the text prompt), effectively integrating images from the instance spaces without producing seams, although it struggles to generate fine details (Fig. (bb). In the following section, we introduce our novel method that combines the strengths of both approaches to achieve superior quality in unconditional canonical data point generation while maintaining performance in conditional generation.
STOCHSYNC: STOCHASTIC DIFFUSION SYNCHRONIZATION
Based on our analysis comparing Diffusion Synchronization (DS) and Score Distillation Sampling (SDS) in Sec. we propose our novel method, Stochastic Diffusion Synchronization, or StochSyne for short, which combines the best features of each method to achieve superior performance in unconditional canonical sample generation. From the perspective of DS, we introduce three key changes in the algorithm.
Maximum Stochasticity in Synchronization. One of the key differences between SDS and previous DS methods is that SDS can be interpreted utilizing maximum stochasticity in the DDIM denoising step (setting 0g = /I 04-1 in Eq. and thus removing the e, term), while earlier DS methods have explored this aspect. We investigated whether maximum stochasticity helps DS achieve better coherence of samples across instance spaces, similar to what is observed in SDS. As the results shown in Fig. [cc), indeed helps remove seams, resulting in much smoother transitions across views. However, we also observe a trade-off between coherence and realism: increased stochasticity leads to greater deviation from the data distribution, producing less realistic images. We present a more detailed analysis of maximum stochasticity on global consistency and realism in Appendix (Sec. D along with experimental results.
Multi-Step Xojz Computation. To resolve the trade-off between coherence and realism, we propose replacing the computation of Xojt from Tweedie's 's formula (Eq. 4). the one-step prediction of the clean sample Xg from x,, with a multi-step deterministic denoising process of DDIM, denoted as (xx)). We observe that a more accurate prediction of the clean samples Xuja at each step along with maximum stochasticity level allows us to achieve both high coherence and realism as shown in Fig. d). Notably, when replacing the computation of Xojz with multi-step denoising, StochSync can also be viewed as iterating SDEdit (Meng et al. [20T]: performing the forward process from X X at timestep (Alg. 4 line 11), followed by the reverse process back to xo (line 12).

---

Published as a conference paper at ICLR 2025
Table 1: Quantitative results of panorama gener- ation using the prompts provided in PanFusion iZhang et al. ). GIQA is scaled by 10s. The best result each column is highlighted in bold, and the runner-up up is underlined.
Table 2: Effectiveness of each components using the prompts provided in PanFusion (Zhang al. 2024a)). GIQA is scaled by 10 The best result each column highlighted in bold, and the runner-up up is underlined.
Method
FID )
IS 1
GIQA
CLIP 1
Td
Max
Impr.
N.O.
FID 4
1
GIQA
CLIP
SDS
96.44
8.21
17.90
30.87
de
Xajt
Views
SDI
143.70
8.08
15.03
29.12
X
80.55
8.65
18.22
30.07
ISM
114.32
8.16
17.08
31.31
V
X
138.82
6.98
15.68
27.95
MVDiffusion
70.49
10.87
18.81
30.79
3
x
x
84.87
7.33
19.06
30.49
PanFusion
93.85
9.90
28.21
78.56
8.54
18.44
30.18
L-MAGIC
59.83
9.12
19.13
29.73
5
117.09
7.56
16.32
28.75
StochSync
57.88
10.02
20.30
31.01
57.88
10,02
20.30
31.01
As result, the loop in line can interpreted not as performing the reverse process but as iterating SDEdit, meaning it does not need to proceed from timestep T to 1. Empirically, we find that stopping the iteration earlier with stop >> 1 provides comparable results while saving computation time. More implementation details and comparisons of inference speed against baseline methods are provided in Appendix (Sec. and Sec.
Von-Overlapping View Sampling, In DS, Xgle is not directly used in the next timestep; instead, it is first averaged in the canonical space (Alg. line 15) and then projected back the instance space (line 10). We note that this modification of Xnjı also results in degradation of realism in the final output. To address this, we propose to sample views at each step without overlaps. Xoje is still synchronized over time, as the set of non-overlapping views newly sampled at each step has overlaps with the views sampled previous steps. In practice, we alternate between two sets of non-overlapping views-one being shift of the other. The result further improved with the non-overlapping views is also shown in Fig. (
Pseudocode and Changes from DS. The pseudocode for our StochSync, incorporating the aforementioned three major changes from DS, provided in Alg. 4 Compared DS (Alg. 2 the computation is omitted due to the use of maximum stochasticity, Tweedie's 's formula is changed multi-step computation ((-) (line 12), and the set of views is not fixed but is sampled without overlaps within the set at each step (line 9). In Alg. 4 the changes are highlighted in red.
Perspective from SDS. From the SDS perspective, StochSyne also be seen implementing three major changes. First, each iteration is performed not with a random timestep / but with a linearly decreasing timestep (Alg. 4 line 8), following the scheduling of the reverse process. At each timestep, multiple views are selected and updated simultancously. Second, instead of reflecting Xoje to the canonical sample z through gradient descent, we fully minimize the 12 loss (line 14). Third, the computation of X e is changed to multi-step denoising (line 12). In other words, St StochSye y can be seen as modification of SDS, designed more closely resemble the reverse process with a decreasing time schedule, while ensuring tighter alignment between the instance space samples and the canonical space sample at each step.
Comparisons to SDS Variants. Recent variants of SDS have proposed changes to certain aspects of SDS, without observing connection the synchronization framework, which have explored for the first time to our knowledge. DreamTime (Huang et al., 2023) suggested decreasing the timestep instead of random sampling. We find that additionally replacing gradient descent with solving a minimization leads to significant improvements. SDI (Lukoianov et al.) 2024) takes the opposite approach from ours, reducing the stochasticity of SDS zero while requiring 6G Since cannot be maintained when views are randomly sampled, it is computed by performing DDIM inversion (Mokady et al., 2023) on Xpjs at every timestep. We empirically observe that this approach is not robust and Irequently fails to converge for panorama and mesh texture generation, as shown in Fig. 2 e). ISM (Liang et al. 2024) also discusses the idea of solving an ODE for Xoj: (multi-step computation) every timestep. but it does not change gradient descent to solving the minimization. In Sec. / we demonstrate the superior performance of ScchSSyn compared to these methods in depth-free 360 panorama generation.
7

---

Published as a conference paper at ICLR 2025
EXPERIMENT RESULTS
In this section, we present the experimental results of StochSync for two applications: 360 panorama generation and 3D mesh texturing. 360 panorama generation is an example of uncon- ditional canonical data point generation (except for text conditioning), while 3D mesh texturing is an example of using depth maps as conditioning. We provide comparisons with baseline methods,
user study results, as well as ablation study results. Appendix, we include implementation details (Sec. BB) details of the user study (Sec. []. and additional qualitative and quantitative results (Sec. [ Extensions of StochSync to additional applications, such as 8K panorama generation and 3D Gaussians texturing, are provided in Appendix (Sec. F).
7.1 360 PANORAMA GENERATION
In the 360 panorama generation, the projection operation f is equirectangular projection, which maps a 360 panoramic image to perspective view images. We specifically use 'Stable Diffusion 2.1 Base' the pretrained diffusion model for all methods, except for the baselines that require finetuned models or inpainting models. We evaluate St ochSSnc on sets of prompts provided by the previous works: 121 out-of-distribution prompts from PanFusion (Zhang et al. 2024a) and 20 CharGPT-gennrated prompts from L-MAGIC (Cai et al. (0224). The results in the rest of this section are for PanFusion prompts, while the results for L-MAGIC prompts are provided in Appendix (Sec. G For evaluation, we randomly sample 10 perspective view images from each panorama and generate the same number of images using the pretrained diffusion model, which serves as the reference set for the evaluation metrics.
7.1.1 COMPARISON TO PREVIOUS WORKS
Quantitative and qualitative comparisons with the baseline methods using PanFusion (Zhang et al. 2024a prompts are presented in Tab. and Fig. 3 respectively. For quantitative evaluations, we report the Fréchet Inception Distance (FID) (Heusel et al. 2018), Inception Score (IS) (Salimans et al. 2016). and GIQA (Gu et al. 2020) to assess fidelity and diversity, as well as the CL.IP score (Radford al. 2 to evaluate text alignment.
As shown in Tab. [ StochSync outperforms SDS (Poole et al., 2023 and its variants, SDI (Lukoianov et 2024) and ISM (Liang al., 2024), by significant margins in all metrics, except for the CLIP score, where ours is still close to the best. Notably, SDI and ISM are not robust and often generate poor outputs, as examples are shown on the left in rows 2-3 of Fig. and more at the end of Appendix (Sec. G).
We also compare StochSyne with finetuning-based methods such as MVDiffusion (Tang et al. 20235) and PanFusion (Zhang et al. 2024a), which finetune pretrained image diffusion model using panoramic images. Due to the lack of large-scale datasets for panoramic images, these finetuning- based methods tend to overfit to the prompts and images used during training. reducing realism for unseen prompts. Hence, our zero-shot method outperforms these methods quantitatively across all metrics, with particularly large margins for FID, except for IS scores where the results are comparable. Qualitatively, our method also demonstrates superior performance compared to theirs, as shown in Fig. (rows 4-5, left). More examples can be found at the end of Appendix (Sec. ().
Lastly, we compare StochSync with the state-of-the-art zero-shot 360 panorama generation method, L-MAGIC (Cai et al. 2024), which uses inpainting diffusion model to sequentially fill a panoramic images. Quantitatively. StochSync outperforms this method across all metrics. Qualitatively, we observe that L-MAGIC often exhibits a "wavy effect" (Brown & Lowe, (007) causing the horizon to appear curved, as shown at the bottom left of Fig. 3 While this geometric distortion may not be fully captured in the quantitative metrics, it significantly detract from the visual quality in terms of human perception. To further evaluate this, we conducted a user study comparing StochSync and L-MAGIC on both the PanFusion prompts and new set of 20 prompts generated by ChatGPT, specifically including the word "horizon". StochSync was preferred over L-MAGIC by 56.20% for the former, with the preference increasing 64.75% for the horizon-specific prompts, demonstrating the superior ability of StochSync to avoid producing curved horizons. Details of the user study are provided in Appendix (Sec. C).
7.1.2 ABLATION STUDY RESULTS
Tab. 2 and Fig. 3 (right) demonstrate the effectiveness of each component of StochSync dis- cussed in Sec. 6 maximum stochasticity (Max σt), multi-step denoising for Xoj: (Impr. Xoje). and
8

---

Published as a conference paper at ICLR 2025
non-overlapping view sampling (N.O. Views). As discussed Sec. DS, represented by SweeTweed- ies (Kim et al. 2024a), generates plausible local images but lacks global coherence across views and thus produce visible seams (row of Fig. 3) With maximum stochasticity, global coherence improves but at the cost of realism (row 2 of Fig.
which is also reflected in the poor quantitative results (row 2 of Tab. 2) Noticeable improvements occur when the computation of Xojz is also replaced with multi-step denoising, (xx:) (row 4 of Fig. 3 and Tab. 2 Finally, the full version of StochSync, using sets of non-overlapping views, produces the most realistic and coherent panoramic images both qualitatively and quantitatively (row 6 of Fig. 3 and Tab. ( Refer to the other rows for additional ablation cases. Note that non-overlapping views require maximum stochasticity, e, cannot be computed when views are not fixed but sampled differently every time.
Table 3: Quantitative results of 3D mesh texturing. KID is scaled by
Metric
Syne- Tweedies
Paint-it
Paint3D
TEXTure
Text2Txx
Sync- Stoch
10 The best result in each row is highlighted in bold, and the runner-
FID CLIP 1
21.76
28.23
31.66
34,98
26.10
22.29
5.69
6.83
2.51
1.31
28.89
28.55
28.04
28.63
27.94
28.57
up is underlined.
7.2
3D MESH TEXTURING
3D mesh texturing is a task where depth map from each view can be used as a condition for image generation, allowing the use of conditional diffusion models (e.g., ControlNet (Zhang et al. 2023 While previous DS-based methods perform well when strong conditions are provided, we demonstrate that StochSync, designed to focus on the unconditional case, provides results comparable to previous DS methods and outperforms other state-of-the-ant texture generation methods.
In our experiments, we follow the experiment setup of SyncTweedies (Kim et al., 2024a) while using the same 429 mesh and prompt pairs. The quantitative and qualitative results presented in Tab. B and Fig. 4 respectively. Note that the results from other baseline methods are sourced from Kim et al. (2024)). In Tab. StochSync outperforms all other baselines across all metrics, with the exception of SyncTweediees our base synchronization framework, which shows comparable results. This demonstrates the versatility of our method, as it can be adapted to applications regardless of whether strong conditional inputs are present. In Fig. 4 Stoch generates texture images with fine details, as seen in the face of the bunny (column 1) and the wood grain patterns of the crate (column 2), whereas Paint-it YYouwang et al. 2202)) leveraging SDS produces images that lack such details. Paint3D (Zeng et al. 2024). which finetunes a diffusion model on the textured mesh dataset (Deikke et al. 2023). fails to capture these details, as seen in the globe (column 4) and the pumpkin (column 6). This aligns with the observation made the 360 panorama generation task, where finetuning on small-scale dataset may result in the loss of rich priors learned by a pretrained diffusion model. Lastly, outpainting-be
methods, TEXTure and Text2Tex (Richardson et al., 20223] Chen et al. 2023a), generate texture images with visible seams due to error accumulation, as shown in the goldfihh (column 7) and the screen of the television (column 8).
Fig. 5 also showcases 3D mesh textures on spheres and tori generated by StochSyne withou depth conditioning, showing the potential for various visual content generation (c.g.ggmme maps).
8 CONCLUSION AND FUTURE WORK
We have introduced St StochSyne, novel zero-shot method for data generation in arbitrary spaces that fuses Diffusion Synchronization (DS) and Score Distillation Sampling (SDS) into the best form for achieving superior performance cases where strong conditioning not provided. Our key insights, based on analyses of the differences between DS and SDS, were to maximize stochasticity in the denoising process achieve coherence across views, while enhancing realism through multi- step denoising for clean sample predictions at each step and sampling non-overlapping views. We demonstrated state-of-the-ard performance depth-free 360 panorama generation and depth-based mesh texture generation.
Limitation and Future Work. Synchronization methods, including ours, face challenges in 3D NeRF (Mildenhall et al., 202T) or Gaussian splat Kerbl et al. (2023) generation, as solving the 12-minimization at each step typically leads to overfitting to individual views when the intermediate images are inconsistent. This issue could be resolved by initializing the 3D geometry with 3D generative models (Hong et al. 2023 Tang et al. 2024), which plan to explore in future work.

---

Published as a conference paper at ICLR 2025
"Desert sunrise silhouettes."
SDS Poole al. 2023)
Syne' Fweddesss Kinn et 2024a)
SDI d Lakoianov al.
2024
Syn:Tweedich Tweedies Max
Sync
ISM (Liang
2024)
SyneTweedeer Impr. Xoje
MVDiffusion Tang et al.
2023b
SyneTwecdisg Max Impr. Xoiz
PanFusion Zhang et
2024a
SyncTweedics + Max da N.O. Views
L-MAGIC Cai et 2024
StochSyne
Figure 3: Qualitative results of panorama generation using PanFusion Zhang et al. 2024a) prompts. Comparisons to previous works are presented in the left column, while the ablation cases are shown in the right column along with StochSyne.
bunay
Crate
Cup
Globe
Pancake
Pumpkin
Goldfish
Television set
SyneTweedies
2024
Youwang al. 2023
Puint3D Zong of all 2004
TEXTure
Richurdson
Text2Tex
StochSync
Figure 4: Qualitative result of 3D mesh texturing. StochSync generates realistic texture images, demonstrating its applicability even in the conditional generation case.
Figure 5: 3D mesh textures spheres and tori generated by StochSync.
10

---

Published as a conference paper at ICLR 2025
ETHICS STATEMENT
StochSyne leverages diffusion model (Rombach et al., 2022) trained on the LAION-5B dataset (Schuhmann et al. 2022). which has been preprocessed to remove unethical content. However, despite these efforts, the pretrained diffusion model may still generate undesirable content when presented with misleading or harmful prompts, limitation that our method also inherits. It is important to acknowledge this risk, as models like StochSync could inadvertently produce biased or inappropriate outputs and should be used with caution. Additionally, StochSync may impact the creative industry by automating parts of the generative process. However, it also offers opportunities to enhance productivity and accessibility to generative tools.
REPRODUCIBILITY STATEMENT
StochSyne uses the 'Stable Diffusion 2.1 Base' (Rombach et al. 2022) for 360 panorama generation and the depth-conditions ControlNet (Zhang et al. 2023) for 3D mesh texturing, both of which are publicly available. We also provide the pseudocode StocSSync in Alg. 3 and the implementation details including hyperparameters in Sec.
B We will also release our code publicly.
ACKNOWLEDGEMENTS
This work supported by the NRF grant (RS-2023-002097233) and IITP grants (RS-2022-11225544 RS-2023-00227592. 00--222--039981177 all funded by the Korean government (MSIT), as well as grants from the ORB-KAIST SketchTheFuture Research Center, NAVER-Inter and the KAIST Undergraduate Research Participation Program.
REFERENCES
Omer Bar-Tal, Lior Yariv, Yaron Lipman, and Tali Dekel. MultiDiffusion: Fusing diffusion paths for controlled image generation. In ICML, 2023.
Matthew Brown and David G Lowe. Automatic panoramic image stitching using invariant features. IJCV, 2007.
Zhipeng Cai, Matthias Mueller, Reiner Birkl, Diana Wofk, Shao- Yen Tseng, JunDa Cheng, Gabriela Ben-Melech Stan, Vasudev Lai, and Michael Paulitsch. L-magic: Language model assisted generation of images with coherence. In CVPR, 2024.
Tianshi Cao, Karsten Kreis, Sanja Fidler, Nicholas Sharp, and Kangxue Yin. TexFusion: Synthesizing 3d textures with rext-guided image diffusion models. In CVPR, 2023.
Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. In International Conference 3D Vision (3DV), 2017.
Dave Zhenyu Chen, Yawar Siddiqui, Hsin- Hsin-Ying Lee, Sergey Tulyakov, and Matthias Nießner. Text2tex: Text-driven texture synthesis via diffusion models. In ICCV, 2023a.
Hansheng Chen, Ruoxi Shi, Yulin Liu, Bokui Shen, Jiayuan Gu, Gordon Wetzstein, Hao Su, and Leonidas Guibas. Generic 3d diffusion adapter using controlled multi-view editing. In SIGGRAPH, 2024a.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. NeurIPS, 2018
Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fantasia3D: Disentangling geometry and appear- ance for high-quality text-to-3d content creation. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 222462225665 2023b.
Zhaoxi Chen, Guangcong Wang, and Ziwei Liu. Text2Light: Zero-shot text-driven hdr panorama generation. ACM TOG, 2022.
Ziyang Chen, Daniel Geng, and Andrew Owens. Images that sound: Composing images and sounds single canvas. arXiv preprint arXiv:2405.1 /2221, 2024b.

---

Published as a conference paper at ICLR 2025
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Osear Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: A universe of annotated 3d objects. In CVPR, 2023.
Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12873-12883, 2021.
Daniel Geng, Inbum Park, and Andrew Owens. Factorized diffusion: Perceptual illusions by noise decomposition. In ECCV, 2024a.
Daniel Geng, Inbum Park, and Andrew Owens. Visual anagrams: Generating multi-view optical illusions with diffusion models. In CVPR, 2024b.
Shuyang Gu, Jianmin Bao, Dong Chen, and Fang Wen. Giqa: Generated image quality assessment. In ECCV, 2020.
Amir Hertz, Kfir Aberman, and Daniel Cohen-Or. Delta denoising score. In ICCV, 2023.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by two time-scale update rule converge to a local nash equilibrium. NeurIPS, 2018.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NeurIPS, 2020.
Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan. LRM: Large reconstruction model for single image to 3d. ICLR, 2023.
Yukun Huang. Jianan Wang, Yukai Shi, Boshi Tang, Xianbiao Qi. and Lei Zhang. Dreamtime: An improved optimization strategy for diffusion-guided 3d generation. In ICLR, 2023.
Dong Huo, Zixin Guo, Xinxin Zuo, Zhihao Shi, Juwei Lu, Peng Dai. Songeen Xu, Li Cheng, and Yee- Hong Yang. Texgen: Text-guided 3d texture generation with multi-view sampling and resampling. In ECCV, 2024.
Oren Katzir, Or Patashnik, Daniel Cohen-On and Dani Lischinski. Noise-free score distillation. arXiv preprint arXiv:2310. 2023.
Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühhlrr and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM TOG, 2023.
Jaihoon Kim, Juil Koo, Kyeongmin Yeo, and Minhyuk Sung. Synetweedies: general generative framework based on synchronized diffusions. arXiv preprint arXiv:2403. 14370, 2024a.
Jeongsol Kim, Geon Yeong Park, and Jong Chul Ye. Dreamsampler: Unifying diffusion sampling and score distillation for image manipulation. In ECCV, 2024b.
Juil Koo, Chanho Park, and Minhyuk Sung. Posterior distillation sampling. In CVPR, 2024.
Yuseung Lee, Kunho Kim, Hyunjin Kim, and Minhyuk Sung. SyncDiffusion: Coherent montage via synchronized joint diffusions. In NeurlPS, 2023.
Yixun Liang, Xin Yang, Jiantao Lin, Haodong Li, Xiaogang Xu, and Yingcong Chen. Luciddreamer: Towards high-fidelity text-to-3d generation via interval score matching. In CVPR, 2024.
Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. Compositional visual generation with composable diffusion models. In ECCV, 2022.
Yuxin Liu, Minshan Xie, Hanyuan Liu, and Tien-Tsin Wong. Text-guided texturing by synchronized multi-view diffusion. arXiv preprint arXiv:2311.12891 2023.
Cheng Lu, Yubao Zhou, Fan Bao. Jianfei Chen, Chongxuan Li, and Jun Zhu. Dom-solver: fast ode solver for diffusion probabilistic model sampling in around 10 steps. In NeurIPS, 2022a.
12

---

Published as a conference paper at ICLR 2025
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver++:: Fast solver for guided sampling of diffusion probabilistic models. arXiv preprint arXiv:2211.00092 2022b.
Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In CVPR, pp. 11461-11471, 2022.
Artem Lukoianov, Haitz Sáez de Ocáriz Borde, Kristjan Greenewald, Vitor Campagnolo Guizilini, Timur Bagautdinov, Vincent Sitzmann, and Justin Solomon. Score distillation via reparametrized ddim. arXiv preprint arXiv:2405. 2024.
Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. SDEdit: Guided image synthesis and editing with stochastic differential equations. In ICLR, 2021.
Gal Metzer, Elad Richardson, Or Patashnik, Raja Giryes, and Daniel Cohen-Or. Latent-NeRF for shape-guides generation of 3d shapes and textures. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 12663-12673, 2023,
Midjourney. Midjourney. httpp::/ http:///wwwwwmmmjooooooo. com/
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing scenes as neural radiance fields for view synthesis. In ECCV, 2021.
Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. In CVPR, 2023.
Jangho Park, Gihyun Kwon, and Jong Chul Ye. ED-NeRF: Efficient text-guided editing of 3D scene using latent space NeRF. In ICLR, 2023.
Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Mildenhall. Dreamfusion: Text-to-3D using 2D diffusion. In ICLR, 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, 2021.
Elad Richardson, Gal Metzer, Yuval Alaluf, Raja Giryes, and Daniel Cohen-Or. Texture: Text-guided texturing of 3d shapes. ACM TOG, 2023.
Herbert Robbins. An empirical bayes approach to statistics. In Breakthroughs Statistics: Foundations and basic theory. Springer, 1956.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In CVPR, 2022.
Tim Salimans, Ian Goodfellow, Wojeiech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. NeurIPS, 2016.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. LAION-SB: An open large-scale dataset for training next generation image-text models. In NeurIPS. 2022.
Yonatan Shafir, Guy Tevet, Roy Kapon, and Amit H Bermano. Human motion diffusion as a generative prior. In ICLR, 2024.
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In (CM1
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In ICLR, 2021a.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In ICLR, 2021b.
13

---

Published as a conference paper at ICLR 2025
Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng. Dreamgaussian: Generative gaussian splatting for efficient 3d content creation. In ICLR, 2023a.
Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. Lgm: Large multi-view gaussian model for high-resolution 3d content creation. arXiv preprint arXiv:2402.05054. 2024.
Shitao Tang, Fuyang Zhang, Jiacheng Chen, Peng Wang, and Yasutaka Furukawa. MVDiffusion: Enabling holistic multi-view image generation with correspondenceawaar diffusion. In NeurlPS, 2023b.
Xiaojuan Wang, Janne Kontkanen, Brian Curless, Steven Seitz, Ira Kemeelmacher-Shiizeemne Ben Mildenhall, Pratul Srinivasan, Dor Verbin, and Aleksander Holynski. Generative powers of ten. In CVPR, 2024a.
Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolific- dreamer: High-fidelity and diverse text-to-3d generation with variational score distillation. In NeurIPS, 2024b.
Seungwoo Yoo, Kunho Kim, Vladimir G Kim, and Minhyuk Sung. As-Plausibee-A--oosiii Plausibility-Aware Mesh Deformation Using 2D Diffusion Priors. In CVPR, 2024.
Kim Youwang, Tae-Hyun Oh, and Gerard Pons-Moll. Paint-it: Text-to-texture synthesis via deep convolutional texture map optimization and physically-based rendering. arXiv preprint arXiv:2312.11360 2023.
Xianfang Zeng, Xin Chen, Zhongqi Qi, Wen Liu, Ziho Zhao, Zhibin Wang, BIN FU, Yong Liu, and Gang Yu. Paint3d: Paint anything 3d with lighting-less texture diffusion models. In CVPR, 2024.
Cheng Zhang, Qianyi Wu, Camilo Cruz Gambardella, Xiaoshui Huang. Dinh Phung, Wanli Ouyang. and Jianfei Cai. Taming stable diffusion for text to 360o panorama image generation. In CVPR, 2024a.
Hongkun Zhang, Zherong Pan, Congyi Zhang, Lifeng Zhu, and Xifeng Gao. Texpainter: Generative mesh texturing with multi-view consistency. In SIGGRAPH, 2024b.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control text-to-image diffusion models. In CVPR, 2023.
Junzhe Zhu, Peiye Zhuang, and Sanmi Koyejo. Hifa: High-fidelity text-to-3d generation with advanced diffusion guidance. In ICLR, 2023.
14

---

Published as a conference paper at ICLR 2025
APPENDIX
A REFORMULATION OF SDS Loss
Here, we show that the SDS loss introduced in Sec. 5.2 of the main paper is equivalent to the original loss presented in DreamFusion (Poole et 2023) up scale. In Sec. 5.2 the SDS loss presented from the perspective of clean samples:
[[ - xo11-111
X
VI 00-16 Xx-1
v or 11-160(2)-1.))
va--1
V va--1
at-l
||e ex(xx-1,))]
(8)
04-1
where the equality in the first line holds from Eq. and is sampled from standard Gaussian, I). Previous works (Kim et al. observation.
2024b Lukoianov et al. 2024) have also made a similar
B IMPLEMENTATION DETAILS
Panorama Generation. We set the resolution of the perspective view images to 512 512, and the panorama to 2, 048 x 4, 096. A linearly decreasing timestep schedule is employed, starting from 900 and decreasing to stop 270, with a total of 25 denoising steps. For multi-step Xoje computation, the total number of steps is initially set to 50, decreasing linearly the denoising process progresses. For view sampling, we alternate between two sets containing five views each, with azimuth angles of [0 72", 144", 216", 288"] and [36", 108", 180", 252", 324%). The elevation angle is set to 0%, and the field of view (FoV) is set to 72
For methods utilizing multi-step Xoje predictions, computing Xoje-1 ((x-1)) in line 12 of Alg. only for the last two steps in the loop of line 8, we leverage the previous Xoje to better preserve the boundary regions. We perform the denoising process while blending the noisy data point as foreground and the previous Xuje as background, as done in RePaint (Lugmayr et al. 2022). For the background mask, we start from the entire region and gradually decrease the regrons over time be close to the boundaries.
3D Mesh Texturing. For 3D mesh texturing, we follow the approach in SyneTweedies (Kim et al. 20244) and use the same image and texture resolutions. We the same number of steps in the 360 panorama generation task with a linearly decreasing time schedule from T = 1, 000 to Toop 270. We use 4 views minimize overlaps between the views. For multi-step Xaje predictions, we use the same refinement mentioned above.
C USER STUDY DETAILS
In this section, we provide details of the user study described in Sec. 7.1.1 of the main paper. We evaluated user preferences across two prompt sets; PanFusion (Zhang et al. 2024a) prompts and horizon-specifit prompts. The study was conducted via Amazon Mechanical Turk (AMT).
Screenshots of the user study are shown in Fig. 6 Participants were presented with two panoramic images (in random order) generated using the same text prompt: one by L-MAGIC(Cai al. 2024 and the other by StochSyne. They were asked to answer the following question: "Which image has better quality, fewer seams, fewer distortions, and better alignment with the given text prompt across the panoramic view?" In each user study, 25 panoramic images were shown in a shuffled order, including five vigilance tests. For the vigilance tests, participants of concatenated 2D square images alongside a ground truth 360 panorama, with the same resolution and question format. For the final results, we collected responses from 50 of 96 participants from the PanFusion set and 59 out of 100 participants from the horizon set, passing at least three vigilance tests. We required participants be AMT Masters and have an approval rate of over 95%.
shown wide image composed
15

---

Published as a conference paper at ICLR 2025
Below are two paro panoramio images generr pererated from the tame text prompt Which one has beftor quality less cears, lower deforions, and bettrr aligmment with givar 1000 prampt acrodd the panoramic vav/? Prompt.
photo
tetinge
(a) Main problem
Thal 7 out
Balbe are too panoramic imagle ganeratac from the same toct rpt.. Wiicch one has better qualit, iss seams, fower distorionns and given prompt "A gerage with cae in
panoramic
Lahinage
Pight men
Vigilanoe tast
Figure 6: Screenshots of the user study. The main test is shown in (a), and the vigilance test in (b).
D ANALYSIS OF MAXIMUM STOCHASTICITY
D. ANALYSIS
Here, provide analysis of maximum stochasticity o; /I at-1 in achieving view consis- tency at the cost quality degradation. To provide clarity in the analysis, we consider a simplified setup where: (1) the instance space consists of single image (N 1, line 9, Alg. 4 (2) the projection operation is replaced with an identity function (line 10, Alg. 4 and (3) the objective function is modified composition of masked (2 losses (lines 7 and 14 of Alg.
Impact of Stochasticity on Consistency. An example of the simplified setup is image inpainting task, where the objective to generate realistic image Xy that aligns with the partial observation y M @ Xp, where M E {0, 1} represents binary mask. To guide the sampling process, the generation is conditioned by replacing M @ Xoje with y.
Under these simplifications, the update rule for becomes:
arg min [I[(1 M) @ xoi=-1)l | @ (z y)1
To analyze the effectiveness of the level of stochasticity on synchronization, we examine the con- vergence rate of measurement error, L(xn()) |M @ X y for two cases: d, and
o, = VI a (Max. σt). respectively. As discussed in Sec. 4 when o, 0, the sampling process becomes fully deterministic. To better illustrate our intuitions, we make two reasonable and straightforward assumptions:
The initial sample xx (NO,T) satisfies L(XoTT) >> and L(g(x))) >
The pretrained noise prediction network c0(-,-) is K-Lipschitz, satisfying (xo( t)
Ex At) < K x, x,-Ail for some constant K.
Under these assumptions, the reformulation of one-step denoising process from the perspective of X yields the following.
1- a1-At (Er -(-
Xok=-Ar
Xooz
(10)
a:-Ai
1 04-A1 At
04-A1 K
Xoit-At -X- Xo/el =
les - EL-Ael <
Xx x1-Ae/=0 =0
(11)
a4-A1
a1-Ai
16

---

Published as a conference paper at ICLR 2025
where the approximation equality holds when Al O. This implies that Xoje-At is largely dependent by the previous sample Xojt: and as a result, the measurement error [(xojt) can remain large even after a few steps of the denoising process, thereby slowing down the convergence of Xoje to y.
On the other hand, when setting o; = -T 0h-1 (Max. o²), Xoje-Ar is no longer dependent on X allowing X2 and -t to differ significantly, even for small At.
This process can be interpreted as resetting the denoising trajectory based on Xoje. allowing the exploration of At that minimizes the measurement error. While it is also true that the newly sampled X could potentially deviate from the desired trajectory and increase ((x))=Ar)) our empirical observations show that, in most cases, it converges the measurement within a few denoising steps.
Impact of Stochasticity on Quality. However, we also observed that sampling with Max. oe degrades the quality of the sample Xo. To address this, we examine the process of sampling Xt-Ar using Max. de. which is described X,- var- + vI Q4-ArE, where (0,
Note that this equation is equivalent to the forward diffusion process described in Eq. 2 exxeept the approximation of Xojt- Unfortunately, the one-step prediction Xoje computed using Tweedie's formula (Robbins) 1956) often deviate from the clean data manifold, sampling process using Max. ae leads to X4- being placed in low-density regions of the noisy data distribution, ultimately degrading the quality of Xg- Inspired by this observation, we note that Xojr should be well-aligned with the clean data X ensure Xujt-Ar to be placed in high-density regions.
This motivates us to incorporate Impr. Xojt. which replaces the one-step predicted Xoje with a more realistic, multi-step predicted Xojc- Additionally, averaging multiple Xgja can introduce blurriness, potentially causing the sample to deviate from the clean data manifold, which leads to the adoption of N.O. Views.
Effect of Increasing the Number of Steps. One might question the validity Impr. Xojz compared to using larger number of steps, as suggested in DDIM (Song et al. (2021a), which demonstrates that increasing the number of sampling steps can improve the quality of generated samples when stochasticity is introduced. However, it is important to note that this claim does not apply to our method, as the DDIM framework focuses on cases where the level stochasticity falls within the
range of or = 0 0
1-ac-1
(DDPM).
1-ax
StochSync sets d. /I- a4-1, utilizing the maximum level of stochasticity. Under this setting, the trend observed in DDIM no longer applies. Specifically, increasing the number of sampling steps does not consistently lead to improved generation quality. In the following, present an informal proof to explain the underlying reason for this divergence.
Statement. Under maximum stochasticity, the diffusion forward process diverges and cannot be approximated by a Stochastic Differential Equation (SDE) as the timestep interval approaches zero.
Proof. Consider the generalized forward diffusion process proposed in DDIM (Song al.. 202
VaLtsi
VI
altsIvai
V1 11-ar ar -o2+s)
X++
Xolr
1-a4
a++11
at -²-At
1-
O1411 aq
OC+AIE,
(12)
where (0, For this process to converge to SDE as At - 0, Lipschitz continuity requires both sides of the equation to approach x,. A necessary condition for this is lim 02 = 0. However, under the maximum level of stochasticity, where o, vI a4-1, this condition is violated. Consequently, increasing the number of timesteps does not refine the distribution but instead causes it deviate further, leading lower-quality unrealistic images.
17

---

Published as a conference paper at ICLR 2025
D.2 EXPERIMENTS
Experiment 1: Image Inpainting. Qualitative results of image inpainting using or 0. Max. de and StochSync are presented in Fig. and Fig. [ The images are obtained by solving the ODE, ((x), initialized from the same random noise XT. Red boxes used to highlight the convergence of Xoje to y. As illustrated, methods with maximum stochasticity (Max. d, and StochSync) converge significantly faster than oN 0, trend also reflected in the measurement emror plot (Fig. ( Additionally, StochSync improves Max. or by mitigating quality degradation in unobserved regions.
"A bowl of cereal with spoon kitchen counter"
000
900
800
700
600
Measurement
Max.
Stochhyre
simple kitchea wita wooden dining table"
1, 000
900
800
700
600
Measurement
de
Max. de
StochSyne
Figure 7: Qualitative result of image inpainting.
18

---

Published as a conference paper at ICLR 2025
"A aubuzban sizeet with houses and clear: blue sky"
1. 000
900
800
700
600
Measurement
Max.
StochSyne
elderly man. sitting
1, 000
900
800
= 700
600
1
Measurement
0,
Max. di
SLochSyne
Figure Qualitative results of image inpainting.
19

---

Published as a conference paper at ICLR 2025
Comparionn
uSER photo dog"
6.1
Number of Steps 10
Number
Number of
Number of
Steps 100 Steps 1, 000
Steps
= 100,00
30
05
700
67
530
800
30
250
Figure 10: Qualitative results of image gener- ation with Max. d₁. Each image is obtained by running different numbers of steps. Sam- pling images with Max. o, for large number of steps fails to generate plausible images.
Figure 9: Measurement error plotted against denoising timesteps. For o, O, the error remains larger than for cases with maximum stochasticity (Max. and StochSync).
Experiment 2: Effect of Increasing the Number of Steps. To validate the theoretical insight on maximum stochasticity diverging for large timesteps, we conduct experiments image generation under maximum stochasticity with varying number of timesteps. We present qualitative results in Fig. 10 which demonstrate that increasing the number of timesteps eventually results in unrealistic images.
E INFERENCE TIME COMPARISON
potential concern about StohhSync could be its computational efficiency, particularly the multi- step computation of Xoje. which might seem to introduce significant overhead. However, we show that this is not the case as our method with optimized hyperparameters achieves a runtime comparable to L-MAGIC Cai et al. (2024) and even outperforms MVDiffusion TTng et al. (2023b). Notably, when integrated with a more efficient ODE solver, our method achieves the fastest runtime for 360 panorama generation, highlighting its computational efficiency.
Experiment Setup. Since StochSyne can be interpreted as an iterative application of SDEdit Meng et al. (2021) across views, the denoising process does not need to run fully to t = 0. Instead, can stop at T Tang > 0, effectively reducing the number of denoising steps. The optimal configuration was found to be Toop 700 with 8 denoising steps, which we denote as StochSync".
Further improvements in efficiency were achieved by incorporating advanced ODE solvers, such as DPM-Solver (DPM-S) Lu et al. (2022a (2 b). This integration, referred to as StochSync* + DPM-S, enables efficient computation of multi-step X with fewer ODE steps, reducing the number of timesteps from 50 to 20 while maintaining comparable output quality.
Experiment Results. Tab. and Tab. we present runtime comparison of StochSync with the baselines for 360 panorama generation and 3D mesh texturing, respectively. For the runtime comparison, the vanilla StochSync was evaluated using the setup described in 7 while baseline methods were tested with their default parameters: 50 denoising steps for MVDiffusion (Tang et al.) 2023b) and PanFusion (Zhang et al. 2024a). 30 steps for Sync Tweedies (Kim et al.) (024a), and 25 steps for L-MAGIC (Cai al. 2024). ) For 3D mesh texturing, the running time results are sourced from SyneT Tweedies (Kim et al. 2024a).
As shown in Tab. 6 StochSyne" achieves run- time comparable to the baselines in 360 panorama generation thanks to carly stopping. Further- more, when combined with DPM-Solver (Lu et al.) 20225) (denoted as S StochSync+tDM-S)) as re- ported in Tab. 6 and Tab. 7 the computation be- comes even faster with only small amount of quality loss. Tab. and Tab. 5 summariie the detailed quantitative scores for 360 panorama gen- eration and mesh texturing, respectively.
Table 4: 360 Panoramas
Method
FID
IS
GIQA
CLIP
StochSync
$7.88
10.02
20.30
31.01
Stochšyne"
47.24
10.80
21.41
31.07
StochSyne' +
47.59
10.43
21.27
31.03
Table 5: Mesh Texturing
Method
FID
KID
CLIP
StochSync
22.29
1.31
28.57
StochSyne+DFW
25.22
2.41
28.60
20

---

Published as a conference paper at ICLR 2025
Table Runtime comparison of panorama gen- eration. The best result in each column high- lighted in bold, and the runner-up is underlined.
Table 7: Runtime comparison of 3D mesh textur- ing. The best result in each column highlighted in bold, and the runner-up is underlined.
Method
Runtime (seconds)
Mothod
Runtime (minutes) J
SyncTweedies
SyncTweodies
183
SDI
920.49
Paint it
21.95
Paint3D
2.65
MVDiffusion
PanFussio
38.33
TEXTure
1.54
I-MAGIC
58.59
Tex22Txx
13.10
StochSyne
149.32
StochSyne"
StochSync
7.61
StochSync +DEM-8
28.05
StochSync+HPX-55
3.36
"Grffiti- covered alleyway with street art murals."
Computo Time of Size Sizeedync Dificent Berations
My Bis 33033
"Devert landscupe with stretches sand dunes."
"Abundoned factory with soil rays Lhrough dusty air. eerie stillnexs."
10
Bevation for Bay
Figure 12: Runtime comparison of NVIDIA RTX A6000 and Intel Gaudi-v2 across three different timestep settings in multi-step Xajı computation.
Figure 11: Qualitative results of 360 panorama genera- tion using Intel Gaudi-v2 with St ScchSSnn..
Results using Gaudi Intel-v2. Furthermore. we showcase qualitative results for 360 panorama generation using Intel Gaudi-v2 in Fig. [T] alongside a runtime comparison with the NVIDIA A6000 in Fig. 12 Our evaluation indicates that the Gaudi-v2 achieves runtimes comparable to those of the A6000.
F ADDITIONAL APPLICATIONS
In this section, provide qualitative results of additional applications of StochSync including high resolution panorama generation (Fig. 13 and texturing 3D Gaussians (Kerbl et al. 2023) (Fig. 14)
High Resolution Panorama Generation. To extend StochSync to high- panorama generation, we modify the original panorama generation setup by narrowing the field of view for individual views and increasing the number of samples, resulting in higher-resolution canonical space sample. However, increasing the number of views introduces the risk repetitive objects appearing in the scene. To mitigate this, employed the refinement technique inspired by SDEdit Meng et al. (202T). Specifically, panorama is first generated using the original setup described in Sec. B The resulting image is perturbed with noise to specific timestep and then refined through the sampling process restarted from this point. This approach effectively addresses repetitive patterns while maintaining high-fidelity details. The qualitative results of 8K panorama generation are presented Fig. T3 demonstrating sharp and visually consistent outputs.
3D Gaussians Texturing. We further demonstrate the capability of StochSyne applications involving complex non-linear projection operations through texturing 3D Gaussians Kerbl et al. (2023). In this experiment, we used Gaussians reconstructed from the Synthetic NeRF dataset Park et al, (2023), updating only their color parameters while keeping their positions and covariances fixed. The results, shown in Fig. demonstrate that StochSync can successfully generate textures of 3D Gaussians.
21

---

Published as a conference paper at ICLR 2025
"Quirky steamourk warkshop Filled with gears and gadgets"
StochSnne
StochSyne w/ 8K Res.
"Rocky desect landscape with Lowering cacti"
SLochSyne
StochSync w/ 8K Res.
"Blegant bailrocd with cryatal chandellers and macble floors"
ochSyne
StochSyne
Figure 13: Qualitative results of high resolution panorama generation using StochSync.
G ADDITIONAL RESULTS
Quantitative Results of 360 Panorama Generation Using L-MAGIC Prompts. The quantitative results of panorama generation using the prompts from L-MAGIC (Cai et al. 2024). well as the
22

---

Published as a conference paper at ICLR 2025
"A luxury chair"
"A mi microphene made of ruby"
"An excavator covered with moss"
"A drum kit made of ruby""
Figure 14: Qualitative results of texturing 3D Gaussians
Kerbl et al. 2023 using StochSyne.
Table Quantitative results of panorama gener- ation using the prompts provided in L-MAGIC (Cai et al. (2024). ). GIQA scaled by 103. The best result in each column is highlighted in bold,
Table 9: Effectiveness of each components using the prompts provided in L-MAGIC (Cai et al. (2024)). GIQA is scaled by 103, The best result in each column is highlighted in bold, and the runner-up underlined.
and the runner-up is underlined.
Method
GIQA
f
Id
Impr. Xoje
FID
IS
GIQA
CLIP
Views
SDS
163.23
5.60
17.41
30.37
SDI
171.69
5.93
16.42
29.33
X
120.19
5.58
19.68
29.34
ISM
197.10
4.92
16.52
29.44
2
V
X
X
178.03
4.76
17.43
28.02
MVDiffusion
31.07
PanFusion
151.60
5.48
18.19
28.46
4
V
v
126.58
5.41
19.34
30.04
L-MAGIC
112.72
5.94
19.73
30.39
169.32
4.74
16.67
28.53
StochSyne
109.41
6.20
20.31
31,22
109.41
6.20
20.31
31.22
ablation study results, are presented in Tab. and Tab. respectively. We observe the same trend as discussed in Sec. [..] where the results with PanFusion (Zhang al. 2024a) prompts are discussed. StochSync generates high-fidelity panoramic images, while L-MAGIC tends to produce panoramas with curved horizons. Refer Sec. G.2 for qualitative results.
Additional Results of 360 Panorama Generation Using Horizon Prompts. Qualitative compar- isons of StochSync and L-MAGIC (Cai et al. 2024) on the horizon-specific prompt set discussed Sec. 7.1.1 are shown in Fig. [5 As discussed in Sec. 7.1.1 L-MAGIC tends to generate wavy panoramas with global distortions, while St SchhSyne produces more realistic panoramic images. This aligns with the results of the user preference test presented Sec. 77..J] where St StochSyre outperforms L-MAGIC on both the PanFusion and horizon-specifee prompts.
Additional Results of 3D Mesh Texturing. Extending the qualitative results presented in Fig. we provide more qualitative results of 3D mesh texturing in Fig. 16
More qualitative results of 360' panorama generation are presented in the following pages.
23

---

Published as a conference paper at ICLR 2025
"A photo of savanna Tanzania with horizon."
"A photo of sunflower field Kansas with herizon."
"A photo tropical island Philippines with horizon."
photo vineyard in Tuscany horizon."
"A photo Patagonin with horizon.
"A photo of the salt Hats Bolivia with horizon." "*
L-MAGIC Cai al. 2024
StochSync
Figure 15: Qualitative comparisons between L-MAGIC (Cai al. 2024 and StochSyne on the horizon-specific prompts.
Apricot
Bookease
Pistal
Polar Bear
Rifle
Shoe
Dumpster
Key
SyneTweedies 2024
Youwnng ot al. 2023
PaintaD Zeng et
Richardson 2023
Text2Tex Chen al 2122a
StochSync
Figure 16: Additional qualitative results of 3D mesh texturing.
24

---

Published as a conference paper at ICLR 2025
G.1
ADDITIONAL. 360 PANORAMA GENERATION RESULTS USING PANFUSION PROMPTS
canyon, sculpted
SDS Poolle al. 2023)
Sync TyneTwee (Kim et all 2024a
2024
SyncTweedies-
ISM Liang et al.
2024
Sync' Tweddess Impr. Xult
MVDifuusion Tann
2023b
SyncTwcodies Impr. Xaie
PanFusion Zhang
2024g
SyncTweedies Max N.O. Views
L-MAGIC Cai al.
StochSyne
"Beneath star-stucded sky, an ancient oak stands sentinel meadow."
SDS (Poole al.
2023)
Syn: SyneTweedies Kim 2024g
Lukoisnov al.
2024
SyneTweedies Max
ISM [Lingg al.
2024)
SyneTweedies Impe. Xujt
MVDillusion (Tang al.
2023h
SyncTweedies Max o, + Impr. Xait
PanFusion Zhang
2024a
SymcTweedies dr
L-MAGIC Cai al. 2024
StochSync
25

---

Published as a conference paper at ICLR 2025
"Desert dunes, endless golden waves."
SDS (Poole al. 2023)
SyneTweedies Kinn et 2024aa
SDI ( LLukoianov al. 2024
Syn:Tweedies Tweedies Max
ISM (Liang 2024)
SyncTweedies Impt. Xojt
MVDiftusion Tang et al.
2023b
Sync Tweedies Max Impr.
PanFusion Zhang et 2024a
SyncTweodier + Max di + N.O. Views
L-MAGIC Cai al.
2024
StochSyne
"Redwood forest, towering tranquility."
SDS (Poole et al. 2023)
Sync Tweedies Kim et al.
2024n
SDI (Lukoisnov al.
2024
Sync' Tweedies Max
Tiang 2024
Sync Sweedweedies.oo Imprr Xoje
MVDifuusion Tang 2023b
SyncTweodies Impr. Xaie
PanPusion Zhang 2024a
Sync Tweediss Max + N.O. Views
L-MAGIC
2024
StochSync
26

---

Published as a conference paper at ICLR 2025
"Moonlit heach, waves whispering secrets."
SDS (Poole al.
2023)
Fweddess Kinn et 2024a
SDI (1 Lakoianov al.
2024
Sync Tweedies Max
(Liang 2024)
SyncTWeodies Impt. Xojt
MVDiftusion Tang al.
2023b
Sync Sweediss Max Impr.
PanFusion Zhang ct
2024a
SyncTweodics + Max di + N.O. Views
L-MAGIC Cai al.
2024
StochSyne
"On distant planet surface, towering crystalline structures against alien
SDS (Poole et al.
2023)
Sync Tweedies (Kim et al.
2024a
SDI LLukoisnov
2024
SyncTweeding Max
(Linng 2024)
Sync Tweneweedies.oo Imprr Xoje
MVDiftusion Tang 2023b
SyneTweedies
Impt. Xaye
Weol
PanPusion Zhang 2024a
SyneTweediss Max N.O. Views
L-MAGIC Cai ctal. 2024
StochSync
27

---

Published as a conference paper at ICLR 2025
"Nestled in canyon, pueblo village stands against the red earth."
SDS Poole al.
2023
SyneTweedee Tweedies Kim et 2024aa
SDT (1 Lulsiianvv al.
2024
SyneTweedies Max
ISM (Liang
2024
SyneTweedies Impt. XoJL
MVDiftusion TTang et al.
2023b
Sync SyncTweedies Max Impr. Xoy
Paniusion Zhang et 2024a
SyneTweodicr + Max di N.O. Views
L-MAGIC Cai at 2024
StochSyne
"On the surface of distant planet, landscape alien rock formations and swirling, multicolored pases."
SDS (Poole et al.
2023)
Sync Tweedies (Km et al. 2024a
SDI Lukoianov al.
2024
Tweddesssss Max
(Tiang
2024)
Sync Impr. Xojt
MVDiftusion Tang 2023b
SyneTweodics Impt. Xaye
PanPusion Zhang
2024a
TweeTwediis Max N.O. Views
L-MAGIC Cai 2024
StochSync
28

---

Published as a conference paper at ICLR 2025
"The interior of historic library, filled with rows of antique books, leather-hound and dust-covered""
SDS Poole et al. 2023)
Syne Tweedies Kinn et 2024a
SDI E Lukoianov al.
2024
Syn: Synedieess Max
ISM (Liang
2024)
Twedies Impr. Xo1t
MV MVDifuusion Tann et al.
2023b
SyneTweedies Max Impr.
PasFusion Zhang ot
2024a
TyecTwesdies + Max di N.O. VieRs
L-MAGIC Cai al. 2024
StochSyne
"Hidlen waterfall, cascading down covered miks tranquil glade."
SDS (Poole al.
2023)
Sync Tweedies [Kim et nl. 2024a
SDI (Lukoisnov ml. 2024
SyneTweedies Max
Giang 2024
Sync Impr. Moje
MVDifuusion Tang
2023b
SyneTweedies
Impr. Xai
Pasion Zhang et 2024a)
Svnc Tweediss Max N.O. Views
L-MAGIC 2024
StochSync
29

---

Published as a conference paper at ICLR 2025
"Surreal desert, mirgge of shimmering heat, dunes stretching endlessly."
SDS (ooole al. 2023)
SyneTweediie Kinn et 2024a
SDI El Lukoianov al.
2024
Tweedies Max
ISM (Liang
2024)
SyndTweedies- Impt. Xojt
MVDifuusion Tang et al.
2023b
Sync Tweedies Max Impr.
PanFusion Zhang et
2024a
Tweeiees + Max di + N.O. Views
L-MAGIC Cai al.
2024
StochSyne
"Alpine meadow, wildflowers swaying mountain breeze, snow-capped pealss embracing panorama- altitdd sanctuary."
SDS (Poole et al.
2023)
Sync Tweedies Kim et al.
2024a
SDI LLukoisnov al.
2024
SyncTweeding Max
Liang
2024)
Sync Impr. Moje
MVDifuusion Tang 2023b
SyneTwoodies Impt. Xaye
PanFusion Zhang 2024a
Tweediss Max N.O. Views
L-MAGIC
2024
StochSync
30

---

Published as a conference paper at ICLR 2025
"Alpine village, snow-covered rooftops, nestled hetween majestic peaks picture-perfect sceme of winter tranquility."
SDS Poole al.
2023
Syne' Twediess (iim et 2024a
SDI (T Luteoianov al.
2024
SyneTweedies Max
ISM
2024
SyacTweedies Impt. Xojt
MVDiftusion Tang et al.
2023b
Syeedweedies Max Impr. Xa1z
PanFusion Zhang et 2024a
SyneTweodics + Max di N.O. Views
L-MAGIC Cai al. 2024
StochSyne
"Inside Aloating above the clouds, suspended by levitating platforms and connected intricate dky bridges."
SDS Poople al.
2023
Sync Tweedies (Km et al. 2024a
SDI (Lukoisnov al.
2024
Synce Tyeediee Max
Ciang 2024
SyncTweedies Impr. Xoje
MVDifuusion Tang 2023b
SyncTweedies Impt. Xay
PanPusion Zhang 2024g
SyncTwoediss Max N.O. Views
L-MAGIC
2024
StochSync
31

---

Published as a conference paper at ICLR 2025
"Desert canyon, ancient rock formations sculptedd by time, vast expanse of temacotta hues an arid symphony of textures."
SDS Poole al.
2023)
Ewediess Kim et
2024a
SDI (1 Lukoianov al.
2024
Syn:Tweedich Tweedies Max
ISM (Liang
2024
SyncTweedies- Impt. Xoj(
MVDiftusion Tang et al.
2023b
Sync Tweedies Max Impr. Xo
PanFusion Zhang et 2024a
Sync Tweedies + Max di N.O. Views
L-MAGIC Cai at 2024
StochSyne
"Stunding cliff, overlooking desert landscupe with towering sand dunes and distant casis."
SDS (Poole et al.
2023)
Sync Tweedies Kim et al. 2024a
SDI (Lukoisnov al.
2024
SyncTweedies Max
(Tiang
2024
SyneTweedee Impr. Xoje
MVDiftusion Tang 2023b
SyncTweodies Impr. Xay
PanPusion Zhang 2024a
Tweediss Max N.O. Views
L-MAGIC Cai 2024
StochSync
32

---

Published as a conference paper at ICLR 2025
G.2
MORE 360 PANORAMA GENERATION RESULTS USING I.- L-MAGIC PROMPTS sky"
SDS Poole et al.
2023)
SyncTweediee (Kim et all 2024a
2024
SyneTweedies
ISM Liang et al.
2024)
SyneTweodies Impr. Xolt
Diffussion Tang et
2023b
SyncTwoodies Max Impr. Xaie
PanFusion Zhang
2024g)
SyneTweedies Max N.O. Views
L-MAGIC Cai al. 2024
StochSync
"Snowy mountain peak view"
SDS (Poole al.
2023)
Sync Tweedies Kim 2024g
Lukoisnov al.
2024
Sync Twediieeees MMx a.
ISM (Liang al.
2024)
SyncTweedies Impe Xajt
MiDiiussion Tang ml.
20238
SyncTweedies Max di + Impr. Xaji
PanFusion Zhang
SymeTTweedies a.
L-MAGIC Cal et 2024
StochSync
33

---

Published as a conference paper at ICLR 2025
"Japanese Zen meditation room"
SDS (Poole al.
2023)
Syne Tweedies Kim et 2024a)
SDI (L Luukoianov al. 2024
SyneTweedies Max
ISM
2024
Impt. Xojt
MVDiffusion (Tang et al.
2023b
SyncTweedies Max Impr.
PanFusion Zhang et
2024g
SyncTweodics + Max di + N.O. Views
L-MAGIC Cai al.
2024
StochSyne
"Sakurn blossom park Kyoto"
SDS (Poole al.
2023
Sync Tweedies Kim et 2024a
SDI ( Luukoiaoov ml.
2024
Sync Tweedies Max
Lingg 2024)
Sync Tweedies Tmprr Moje
MVDiftusion Tang 2023b
SyncTweedies Impr. Xay
PanPusion Zhang 2024g
SyneTwes Max N.O. Views
L-MAGIC
2024
StochSync
34


Reviews:
[{"review_id": "N4zQjgGtiO", "reviewer": "Reviewer_yJqb", "summary": "This manuscript proposes a new diffusion synchronization method for panorama/texture generation (or on arbitrary surfaces) using pretrained image diffusion models. The proposed method combines the strengths of SDS and previous synchronization method by adding more stochasticity, using multi-step x0 sampling for synchronization, and using non-overlapping views. Both qualitative and quantitative results seem to demonstrate its strengths on panorama generation and mesh texturing tasks.", "questions": "I do not have any confusion about this manuscript. For rebuttal, please address the weaknesses listed above.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The contributions seem technically sound to me. As someone who works on relevant topics, I believe that adding stochasticity and using multi-step sampling should be better than vanilla synchronization without any doubt (and I have not yet read another paper pointing this out clearly). Also, using non-overlapping views is an interesting design choice.\n\n- The paper makes a good effort at comparing existing methods and the proposed one. The algorithms are very clearly presented, showing the similarities and differences between StochSync, SDS and vanilla synchronization. The writing is also clear in general.\n\n- Experiments look good to me. The contributions of each component are clearly shown in Table 2 and Fig. 2. The results are strong on both panorama and texture generation tasks. Despite the weaker texture generation metrics compared to SyncTweedies, StochSync seems to produce better details, which aligns with my expectation.", "weaknesses": "- I feel that this manuscript could have been a lot better with more in-depth theoretical analysis rather than just empirical results. Why is more stochasticity better? Does non-overlapping view sampling implies strong output correlation between overlapping views which degrades the distribution? Non of these important questions are explained in more depth, no even with some basic intuitions.\n\n- Inference time is not given, which is a very important factor when evaluating these models. Using multi-step computation is clearly more expensive than vanilla synchronization methods.\n\n- Some evaluation metrics might not be proper for the problem. FID and KID metrics compare the generated distribution with a reference. But when using pretrained models for zero-shot adaptation, it's hard to define a standard reference dataset. IS also seems not perfect since it's originally designed for evaluating generative models on categorical data.\n\n- I find some explanation of the empirical result not satisfactory. For example, in L316, I don't think it's simply the increased stochasticiy that worsens the distribution - rather it could be the case that increased stochasticiy effectively makes the sampling time step larger, and should be compensated with multi-step sampling to restore the distribution.\n\n- As the authors have pointed out, a clear limitation of this method is the inability to generate 3D content such as NeRF. However, the explanation of overfitting seems problematic to me as SDS clearly doesn't have such issue.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gP2439pFe7", "reviewer": "Reviewer_DvFj", "summary": "This paper presents a novel method called StochSync for zero-shot image generation in arbitrary spaces, such as 360° panoramic and 3D mesh textures, by leveraging pretrained diffusion models. StochSync builds on two existing approaches—Diffusion Synchronization (DS) and Score Distillation Sampling (SDS)—by combining their strengths: the coherence of DS and the robustness of SDS in weak conditioning scenarios. Experimental results demonstrate that StochSync produces high-quality, seam-free images in various geometries, outperforming or matching zero-shot and fine-tuned methods across multiple evaluation metrics.", "questions": "1. As video diffusion models contain more spatial connection information across different views, can video diffusion benefit such tasks?\n\n2. As PBR materials play an important role in 3D mesh texturing, can StochSync also perform well in PBR-like texturing?\n\n3.  While the paper shows impressive results for unconditional cases, can the authors provide more insight into how StochSync performs under conditional scenarios, particularly in comparison to methods that excel with strong conditioning (e.g., DS methods with depth maps)?\n\n4. Since generating high-resolution outputs is often challenging in diffusion models, could StochSync be adapted or scaled for ultra-high-resolution generation (e.g., for 8K textures or VR applications)?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The authors clearly illustrate their motivation in combining DS and SDS, effectively balancing stochasticity and coherence.\n\n2. StochSync can enhance image generation quality in arbitrary spaces without the need for fine-tuning. The method is also applicable across a variety of geometries, extending the capabilities of pretrained diffusion models beyond their typical square-space applications.\n\n3. The paper provides detailed ablation studies, elucidating the contribution of each component of StochSync, such as maximum stochasticity, multi-step denoising, and non-overlapping view sampling.", "weaknesses": "1. The contribution is limited as it is more likely to be an A (DS) + B (SDS) framework although the authors conduct theoretical analysis. \n\n2. The paper does not discuss computational efficiency in detail. Given the complex multi-step nature of StochSync, the method may struggle to achieve real-time performance, limiting its applicability in interactive or real-time systems. The authors should provide more detailed discussions of time efficiency as SDS-like approach is very time consuming.\n\n3. As multi-view plausibility and consistency is quite significant in 3D objects, the paper only provides a single view of the 3D mesh.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CpYYTnXuwI", "reviewer": "Reviewer_8qis", "summary": "The paper proposes a new method for training-free generalization of image diffusion to generation in more general spaces, such as 360 panorama and 3D mesh textures. The main idea is to replace the one step denoising step in diffusion synchronization with multi step denoising on non-overlapping views. Qualitative results show that it can generate more realistic results with fewer artifacts such as seams in panorama.", "questions": "1. Regarding non-overlapping view sampling for 3D meshes, do you only sample 2 views for each iteration? It seems impossible to sample more than 2 views without mutual overlap.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The idea of using multi-step backward equipped with non-overlapping view sampling is novel.\n\n2. The presentation is clear and detailed, drawing connections of the proposed method to existing methods such as SDS, diffusion synchronization and SDEdit.", "weaknesses": "1. Non-overlapping view sampling seems to be an essential part of the algorithm. If the views are overlapping, each step involves blending several multi-step denoised images, which does not make sense to me. In diffusion synchronization methods this problem is not that big because each blending step only involves images with one-step denoising. However I do not find any ablation study of non-overlapping view sampling, and do not know what kind of artifact it will introduce if we drop this component.\n\n2. Non-overlapping view sampling can be a limitation in some cases, e.g. panorama smaller than 360 degree.\n\n3. Although the paper claims the method can generate for \"arbitrary spaces\", the two tasks shown in the paper (360 panorama and mesh texturing) have relatively simple \"rendering\" functions f. In fact, they are both linear (in terms of the underlying parameters) and so projecting and blending multiple views in the same space is easy. But how about a nonlinear rendering function such as NeRFs? I guess it will be a challenge compared to other methods like SDS because different views are fully denoised to potentially completely different images (especially in early stages), and blending them over time with a complicated rendering function is not easy.\n\n4. It can be much slower than the alternatives because of the multi-step backward process G at each step.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)