Seeds of Structure: Patch PCA Reveals Universal Compositional Cues in Diffusion Models
Authors: Qingsong Wang
Zhengchao Wan
Mikhail Belkin
Yusu Wang
Venue: NeurIPS 2025 poster ()
Keywords: Diffusion models
Principal Component Analysis
Generative model
Primary area: deep_learning
TL;DR: Diffusion noise seeds contain universal compositional blueprints that persist across datasets, enabling our Patch PCA framework for zero-shot structure control.

Abstract: Diffusion models transform random noise into images of remarkable fidelity, yet the structure of this noise-to-image map remains largely unexplored. We investigate this relationship using patch-wise Principal Component Analysis (PCA) and empirically demonstrate that low-frequency components of the initial noise predominantly influence the compositional structure of generated images. Our analyses reveal that noise seeds inherently contain universal compositional cues, evident when identical seeds produce images with similar structural attributes across different datasets and model architectures. Leveraging these insights, we develop and theoretically justify a simple yet effective Patch PCA denoiser that extracts underlying structure from noise using only generic natural image statistics. The robustness of these structural cues is observed to persist across both pixel-space models and latent diffusion models, highlighting their fundamental nature. Finally, we introduce a zero-shot editing method that enables injecting compositional control over generated images, providing an intuitive approach to guided generation without requiring model fine-tuning or additional training.

Seeds of Structure: Patch PCA Reveals Universal Compositional Cues in Diffusion Models
Qingsong Wang
Zhengchao Wan Department of Mathematics University of Missouri Columbia, MO 65211 zvandmissouri.
Halıioğlu Data Science Institute University of California, San Diego
La Jolla, CA 92093
qswang920gmail. com
Mikhail Belkin
Yusu Wang
Halıcoğlu Data Science Institute
Halıooğlu Data Science Institute
University California, San Diego
University of California, San Diego
La Jolla, CA 92093 mbelkin@ucad.ccm edu
La Jolla, CA 92093
yusuvangOucsd.com edu
Abstract
Diffusion models transform random noise into images remarkable fidelity, yet the structure of this noise-to-image map remains largely unexplored. We inves- tigate this relationship using patch-wise Principal Component Analysis (PCA) and empirically demonstrate that low-frequency components of the initial noise predominantly influence the compositional structure of generated images. Our analyses reveal that noise seeds inherently contain universal compositional cues, evident when identical seeds produce images with similar structural attributes across different datasets and model architectures. Leveraging these insights, develop and thcoretically justify a simple yet effective Patch PCA denoiser that extracts underlying structure from noise using only generic natural image statis- tics. The robustness of these structural cues is observed to persist across both pixel-space models and latent diffusion models, highlighting their fundamental nature. Finally, we introduce a zero-shot editing method that enables injecting compositional control over generated images, providing intuitive approach guided generation without requiring model fine-tuning or additional training.
Introduction
have emerged as a powerful class of generative models, achieving remarkable success in image synthesis 26 9 and more [32 [4] These models operate by gradually transforming random noise into structured data through an iterative denoising process. Within the image generation domain-one our focus in this work the noise-to-image map exhibits striking consistency across different architectures and training procedures (37], suggesting intrinsic patterns underlying this transformation that transcend specific implementation details. Understanding these patterns, even approximately, has significant practical value. Recent work (23 19 36 has shown that choices of initial noise seeds significantly influence the alignment between generated images and text prompts, particularly regarding global structure and composition. However, precisely how specific noise patterns influence particular image characteristics remains an open question with substantial implications for controlled generation and image editing.
Diffusion models 27
13 29
In this paper, investigate the influence of initial noise generated image structure through patch- wise Principal Component Analysis (PCA) framework [5] 77. Using this approach, we empirically
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

demonstrate that the eigenspace identified by patch principal components positively correlates with principal variation directions of the toise-to-image map. We focus on the low- -wwfrequency components identified by patch PCA, which in the image patch domain correspond to illumination and major structural lines, contributing to the perceived composition of an image.
Notably, when the low-frequency components of the initial noise are fixed and only high-frequency components are perturbed, the generated images exhibit minimal structural changes. This suggests that low-frequency noise components predominantly determine the compositional structure in diffusion generation. Additionally, our empirical analyses reveal that noise seeds inherently contain universal compositional cues, so images generated from identical noise seeds across different datasets exhibit similar compositional attributes, evidenced through visual examples and quantitative metrics. This finding helps explain phenomena such as the observed variability in text-to-image generation when aligning prompts with global structure and composition across different noise seeds; the seeds themselves, while random, carry distinct instantiations of these underlying compositional templates.
Proposed Methods. The robustness of these compositional cues enables us to develop a simple yet effective Patch PCA denoiser that extracts underlying structure from noise using only generic set of natural images. This approach theoretically justified by Theorem 5.1 where we prove that the Patch PCA denoiser is optimal for the diffusion model loss when the denoiser calculates each pixel's output based solely on local motch-common in convolutional architectures- and the patch distribution is Gaussian, reasonable assumption for small natural image patches [40]. Despite its simplicity, images generated by our Patch PCA denoiser exhibit structural similarities to those from sophisticated neural networks, though less visually refined.
Structural similarities become increasingly apparent when projecting onto leading PCA subspaces, demonstrating these compositional cues primarily reside in low-frreg now-frequency components. These findings extend to latent diffusion models like Stable Diffusion (26). where the same structural hierarchy persists despite operating in a compressed latent space. This cross-moded consistency reveals noise seeds contain intrinsic compositional cues that transcend specific architectures, representing fundamental properties of the generative process itself.
Leveraging these insights, we develop a zero-shot Patch PCA based noise editing method that selectively interpolates noise representations with reference images within specific PCA frequency bands. Our approach enables structural control without model fine-tuning or additional training. providing an efficient method for guided image generation by directly manipulating the compositional blueprint in the noise.
Related Work
Noise Seeds and Composition in Diffusion Models. Recent studies show that initial noise seeds significantly influence compositional structure in diffusion diffusion-generated images, with certain seeds consistently producing better-aligned results for specific prompts 36). This has led to optimization methods for finding high-performing seeds 39 19. While these works establish the importance of noise seeds, the underlying mechanisms of how seeds encode compositional information remain an open area for exploration. Our work contributes to this understanding through patch PCA analysis, demonstrating that compositional cues predominantly reside in low-frequency noise components, propose a zero-shot editing method that can guide the composition of generated images.
Inductive Bias in Diffusion Models. Recent works on diffusion models have explored various aspects of inductive bias. Studies like (20 34] observe linearity in well-trained diffusion models and develop closed-form approximations under Gaussian assumptions, while (24 [5] examine these
models through local image patches, arguing that convolutional architectures constrain each pixel's output to depend only on its surrounding patch. Our work builds upon these insights but takes different cirection-inneeee
of seeking optimal theoretical surrogates for specific datasets, we aim to identify universal compositional structures that generalize across diverse data distributions. By leveraging the observation that local patches follow Gaussian distributions more accurately than full images [40), we extract fundamental compositional cues shared across different domains without dataset-specific optimization. This approach allows us to analyze and manipulate compositional elements embedded in the noise-to-image map that persist across varied model architectures, enabling practical applications like shoo-shot noise editing without requiring dataset-specific tuning training.

---

Preliminaries
Diffusion Models and Denoisers. Diffusion models generate data by reversing gradual noising process. When parameterized by noise-to-signal ratio a as in Song et al. (28), the reverse process follows the ODE:
/xo/do- (D σ) xo)/o
(I)
where p NO,0021 is the data distribution convolved with Gaussian noise and D(x,, σ) is the denoiser. A neural network is typically trained to approximate the denoiser:
xx) (Xx,, - =
(2)
and sampling performed by integrating Equation starting from a random noise.
PCA and linear optimal denoisers. For data distribution p(x) in R with mean μ and bounded covariance Σ, we can decompose the covariance matrix Σ into its eigenvalues and eigenvectors, E = A.u:u?" where X; are the eigenvalues and u, are the corresponding eigenvectors. This is the principal component analysis (PCA) of the data distribution and the eigenvectors are called the principal components. It shown in [20] Theorem 1] that the optimal linear approximation of the denoiser noise level is given by:
A μ4 u.)..
DPCA 0) Σ
A + o2 2x
(3)
which performs scaling along the principal components of the data distribution.
Driginal
0:45
2::4
[3:40]
(4:00)
[6:48
16:00
CIFAR1O
AFRIQ IMAGENET
0.15
MSE
0.10
Original
00:83
12:5
B
5::5
IG
16-88
0.05
0.00
10
20
25
Start index subspace range [n. 48]
MSE across frequency bands perturbations.
(b) ImageNet (top), AFHQ (bottom)
Figure I: Frequency band perturbation analysis. (a) MSE between original and perturbed-noise generated images across frequency bands. When perturbing including low-frequency bands, the MSE is significantly higher than when only from ImageNet (top) and AFHQ (bottom): original image followed by perturbations with full frequency bands [nz, 48] where n 0, 2....... 16. Three different perturbations are shown for each frequency band.
bands are perturbed. (b) Visual examples
ODE sampler and the noise-to-image flow map. The ODE sampler 228
popular for its simplicity and effectiveness, is deterministic. For a convergent Ordinary Differential Equation (ODE), the final image uniquely determined by the initial noise. This relationship defines a mapping, known as flow map, from the noise space to the data manifold. The convergence of the ODEs in diffusion models, and the existence of flow map, have been established in recent studies [12 33.. The flow map imposes a structure on the noise space. For example, for simple dataset of two points noise space into two regions, with all noises from
1- C IR, the flow map partitions negative axis mapped to -1 and all noises from the positive axis mapped to 1. Hence, the final sampled points are fully determined by the sign of the initial noise. It is of great interest to find similar structure in the noise space of image diffusion models.
3

---

4 Probing the Noise-to-Immage Map through Patch PCA filter
In this section, we probe the noise-to-image map in pre-trained diffusion models by analyzing how perturbation in initial noise influences the generated images. Our diagnostic tool is Patch PCA, which effectively captures image statistics and has been shown to be valuable for image denoising tasks [2].
Methodology: We first compute a generic patch PCA covariance matrix using 4 x 4 patches extracted from 10,000 randomly sampled images from the ImageNet dataset and compute the covariance matrix with its eigen seeenddcompostionn In fact, for small patches (size less than 16 x 16), the covariance matrices computed from differently collected images are almost identical; see Figure ( in the Appendix for an ablation study. With 3 color channels, each image patch is a 4 x 4 X 3 tensor, and we obtain 48 wewenaalllleigeeeeeeee pairs from the follow a characteristic power-law distribution typical of natural images, with the leading eigenvectors representing color/illumination variations, followed by edge detectors and then higher frequency components capturing refined textures (see Figure [0a and Figure T0b in the Appendix). We conduct experiments using pre-trained EDM models [17] on diverse datasets: ImageNet [ (64x64), FFHQ, AFHQ, and 10 [8]] We fix 128 random noise initializations and generate reference images using the pre-trained models. For each noise initialization, and for each frequency band [n, 48] (where ranges from 0 to 48), we generate 100 perturbed variants by subdividing the noise tensor into non-overlapping 4 x 4 patches and resampling the identified frequency bands [nz, 48] on each patch; see Algorithm 2 for the details. This ensures independence and preserves the standard Gaussian distribution due to the orthogonality of the PCA basis. We then measure the mean squared error (MSE) between images generated from the perturbed noise and the original reference images.
As expected, the eigenvalues
The analysis in Figure revells a clear pattern: (small n) induce substantial changes in the generated images, while high-frequency perturbations (large n) have minimal impact. Specifically, when n exceeds 20, the MSE remains consistently low (below 0.01, with pixel values in [0, 1)), and the generated images in Section 3 are visually indistinguishable from the originals. This finding is indicative that the crise-to-image map in diffusion models preserves a hierarchical structure aligned with the patch PCA decomposition: freffreqeency components control fundamental aspects such as illumination and global layout, while high-frequency components primarily contribute texture and fine details.
Results:
perturbations in low-frequency components
Cross-dataset compositional resemblance: We observe that images generated from identical noise seeds across different datasets exhibit similar structural patterns. In Figure [b] goldfish (ImageNet) and fox (AFHQ) generated from the same seed share similar lighting patterns, with matching bottom- left highlighting and bottom-right edge shapes. This compositional correspondence persists across datasets (see Figure 2a] first three rows). To quantitatively validate this visual similarity, we compute both MSE and structural similarity index measure (SSIM) (35] between pairs of images generated from identical noise seeds across different datasets. SSIM is particularly valuable for this analysis because, unlike MSE which measures pixel-level differences. SSIM captures structural similarities by considering luminance, contrast, and structure; see Appendix AA.2 for for the detailed definition. This makes SSIM especially suited for detecting compositional resemblances that may not be apparent in pure pixel-wise comparisons. As shown in Table [ both MSE and SSIM scores show significant closeness for seed-paired images (SSIM: 0.423-0.460 -099; MSE: 0.032-0.041) compared to randomly paired images (SSIM: 0.054-0.065; MSE: 0. 0.128-0.1366 136). This confirms that the generated images share structural properties when derived from the same noise initialization across different datasets, providing strong evidence that the initial noise contains universal compositional cues that transcend dataset boundaries and training domains. We include ablation studies on ODE sampling schedules, network architectures (U-Net versus bransformer-base)), and flow-matching variants in Appendix A.2 where we consistently observe the same seed-dependent structural pattern.
5 Uncovering Universal Compositional Cues Through Patch PCA Denoiser
Our cross-dataset experiments revealed consistent compositional patterns when using identical noise initializations. This suggests that initial noise may contain universal compositional cues that transcend specific datasets. We now formalize and validate this hypothesis by directly extracting these compositional elements using our patch PCA framework.

---

Table Image Similarity Between Networks and Patch PCA. SSIM and are computed between images generated from identical noise seed and randomly paired noises across different architectures. Same-seed images show significantly higher similarity (higher SSIM and lower MSE scores) compared to random pairs.
SSIM (higher is better)
MSE (lower better)
Network Pair
Same Seed
Random
Same Seed
Random
ImageNet vs FFHQ
0.423 0.087
0.065 0,040
0.041 + 0.017
0.136 + 0.055
ImageNet vs AFHQ
0,447 + 0.097
0,062 0.039
0.038 = 0.018
0.130 = 0.055
FFHQ vs AFHQ
0.469 0.074
0.054 0.041
0.032 = 0.012
0.128 + 0.046
Patch PCA vs ImageNet
0.474 0.137
0.031 0.027
0.049 + 0.025
0.115 + 0.037
Patch PCA vs FFHQ
0.473 + 0.077
0.029 0.033
0.045 = 0.014
0.114 = 0.027
Patch PCA VS AFHQ
0.548 0.084
0.029 + 0.034
0.036 = 0.014
0.104 = 0.029
Seed
Seed
Swed
Swed
Seed
Seed
Seed
Sead
Seed 0
Seed 1
Seed 2
Seed 3
ImageeNt
InbageNet
1731
F7310
AF3I
AFEE
Patch PCA
Patch PCA
(a) Generated images
(b) Top-1 subspace projection
(c) Top-2 subspace projection
(a) Images from identical seeds on ImageNet, FFHQ, AFHQ, and Patch PCA (bottom). (b) Top-1 projection revealing consistent
Figure 2: Universal compositional cues across datasets and models.
illumination patterns. (c) Top-2 projection with additional similar color patterns.
Patch PCA Denoiser. We propose a simple yet effective Patch PCA denoiser that operates directly on noisy image Xx, the output of the diffusion model at time step a reads
D(x, o) ((DDpA(P(xx)..))
Here P represents the patchification operation that extracts overlapping p X p where p is the patch size and с is the number of channels, patches with necessary padding from the input image X, centered on each pixel. DPCA is the linear PCA denoiser applied to each patch p; in P(x,) via
DPCA μ Σy²n X,
Ay (p. μe, u,)u, where is the patch and {u,, Ay} are the eigenvectors and eigenvalues of the patch covariance matrix computed from a generic set (clean) images and fixed throughout the experiments. 2/ is the reconstruction operation that uses the center pixel of each patch to reconstruct the image. This linear approximation utilizes the idea that compared with the whole image, the patches are closer to being Gaussian distributed, where the linear denoiser optimal. The patch PCA denoiser can be efficiently implemented using a convolutional layer with P p x kernel. In our experiments, we found the approach robust to different patch sizes ranging from 5 to 31, with samples generated from the same noise seed showing consistent results in terms of various metrics; see Appendix A.3 3.3 for detailed ablation studies. We use the Patch PCA denoiser in the DDIM sampling 28
(see Equation
to generate images.
Note that convolutional backbones such as CNNs or U-Nets constrain each output pixel to depend only on fixed-ize local patch. Building on the analytic framework of [15,, we show that-unte under class is nothing but exactly the
Gaussian patch prior- thhor-hhe/ best denoiser within this patch PCA denoiser; see Theorem A.2 in the Appendix for more formal statement and proof.
Theorem 5.1 (Patch PCA denoiser is optimal (Informal)). Assume that the patch distribution follows a Gaussian law with mean and covariance E, and candidate denoiser must compute every pixel from its own patch only. Then the minimizer of the standard diffusion (denoising) loss (ef. Equation ) over all such candidate denoisers is exactly the patch PCA defined Equation 44 in most pixels (exluding pixels close the boundary of image).
5

---

Extracting Universal Compositional Cues. We extract patches from randomly sampled images from the ImageNet and compute the covariance matrix of the patches to construct our Patch PCA denoiser, which in turn is used in DDIM sampling (8
(see Equation (T) to generate images. The resulting images, shown in the bottom row of Figure 2a while not as visually refined as neural network outputs, display remarkable compositional similarities when compared with neural network generations from identical noise seeds. The Patch PCA denoised images exhibit fundamental illumination patterns and structural lines that closely match those in their neural network counterparts. This suggests that our simple patch linear approach effectively captures the underlying compositional cues embedded the noise seeds.
To quantitatively validate that our Patch PCA denoiser captures the same underlying compositional information sophisticated diffusion models, we perform systematic comparison of images generated from identical noise initializations. The results, presented in the bottom three rows of Table [ reveal that Patch PCA denoised images achieve substantially higher SSIM scores with images when sharing the same noise seed (0.474-0.548) compared to random pairings (0.029-0.03)) Notably, these similarity values exceed even those between diffusion models trained on different datasets (0.423-0.469), providing strong statistical evidence that our linear approach successfully extracts universal compositional information encoded in the noise. The MSE also shows significantly lower error between same-seed pairings (0.036-0.044 0.049) compared to random pairings (0. (0.104-0.1) 0.10..1.... 15). Visual inspection in Figure 2a confirms that common illumination gradients, dominant structural lines, and color distribution patterns remain consistent across all images derived from the same seed, regardless of whether they were generated by comain-speciffe neural networks or our domain-agnostic Patch PCA method. This suggests that our approach effectively extracts the compositional cues that diffusion models inherently follow while neural networks trained different datasets add domain- -spcii- refinements and details.
Projection Dimeusions
Projection
ImagsNit
Sinilarity (6SD))
Moua Squared Ervor (MSE)
0.00
0.6
APHO
0.5-
0.00
0.4
0.m0
0.8-
0.02
0.2
Imsgehost
Number PCA Components
Number PCA Components
(a) SSIM scores for the projections
(b) MSE scores for the projections
Figure 3: SSIM and MSE scores for the projections of the generated images from the same noise seed across different datasets. The scores improve when the images are projected onto subspaces with leading eigenvalues, indicating that the patch PCA model captures the similar structural elements across datasets that are preserved in owwffreuenccy components.
Amplifying Compositional Cues Through Subspace Projection. To further investigate and visual- ize the compositional information encoded in the noise, we examine the lower frequency projections of the generated images. For each generated image by neural networks or the Patch PCA denoiser, we first extract patches from the image, project the patches onto the leading eigenvectors of the covariance matrix, and then reconstruct the image from patches by folding back the patches.
The visualizations in Figure reveal that restricting to just the first principal component preserves key illumination patterns and major structural lines while climinating texture and fine details. The compositional resemblance across images from the same noise seed amplified in this highly restricted representation. When expanding to include the second principal component (Figure 2c color variations emerge while maintaining coherence across datasets and the Patch PCA model.
We quantify this phenomenon in Figure Baland and Figure 3b where both SSIM and MSE metrics show systematic improvement as images are compared in these increasingly restricted representations. This progressive enhancement in similarity confirms that the essential compositional information concentrated in just the first few PCA components, corresponding to the most basic structural elements of the images. These findings complement our earlier perturbation experiments in Section

---

providing compelling evidence that diffusion models across different datasets follows fundamental compositional structure in the low-dimensional components of noise identified by the patch PCA.
(a) Generated images
(b) Top-1 subspace projection
(c) Top-S subspace projection
Figure 4: Visualization of the generated images from Stable Diffusion (top row) and Patch PCA model (bottom row) using uncurated seeds 0-3. The first column shows generated images from the same noise seed across different datasets, while the second and third columns show patch PCA denoised images projected onto the top-1 and top-5 subspaces, respectively. The consistent structural similarities demonstrates the effectiveness of our patch PCA denoiser in latent diffusion setting.
Patch PCA with Latent Diffusion Models We now examine whether our findings extend to latent diffusion models, which operate in a compressed latent space rather than pixel space. We focus on Stable Diffusion (V1.4) 26.. adapting our patch PCA framework its VAE latent space.
Our adaptation involves extracting p X patches from encoded ImageNet samples (shape [4, p. pl from latent representations of dimensions [4, 64, 64)), computing the covariance matrix, and performing eigen- degenddccoppoitionn. When decoded back to pixel space, the leading eigenvectors (Figure []] in Appendix A.3) reveal a striking phenomenon: they mirror the patterns observed in pixel-space models, with initial components corresponding to colorfillmination variations, followed by edge detectors at higher frequencies. This cross-domain consistency suggests the VAE decodes the latent space patch eigenvectors into pixel space patch eigenvectors, which could be helpful for preserving structural hierarchies across representation domains and expanding the applicability of our patch PCA approach to various diffusion architectures. This perspective is further validate by the empirical observation in Table 7 in Appendix AA.3 where the encoded pixel-patch and latent patch top-k eigenspaces are closely aligned (mean cosine of principal angles = 0.93 for top 100 subspace).
We upply our Patch PCA denoiser from Equation (4) to Stable Diffusion's 's latent space and visualize the results in Figure 4 for four uncurated seeds (0-3). The first column shows standard Stable Diffusion generations (with empty prompt) while the second and third columns display pateh-based projections onto the top-1 and top-5 subspaces, respectively. Consistent with our pixel-space findings, projections onto the top-1 subspace isolate illumination and major structural lines, while projections onto the top-5 subspace preserve additional color variations and some edge details. The strong visual alignment between generated and patch PCA denoised images confirms that noise seeds encode consistent compositional elements regardless of whether generation occurs in pixel or latent space.
Quantitatively, SSIM scores between Stable Diffusion and patch PCA images are significantly higher for seed-paired (mean SSIM 0.42) comparisons than random pairs (mean SSIM 0.18) in Figure [5a The SSIM scores increase when projecting onto lower-dimensional spaces, reaching 0.816 for top- projection (Figure (5)). Notably, we observe a slight dip at the top-2 projection due to color tone differences between the models, with Stable Diffusion exhibiting red-brownish tone not present in the patch PCA projections (Figure [3). These findings suggest that similar compositional structures may be encoded in noise seeds across both pixel-space and latent diffusion models, indicating that low-frequency noise components play an important role in determining structural elements in different model architectures.
6 Zero-shot Patch PCA based Noise Editing
Our analysis reveals that fow-frequency components of initial noise significantly influence the compositional structure of cettussionnnnnertte images. Building on this insight, propose shot editing method that directly manipulates noise through patch PCA frequency band filtering, enabling compositional control without requiring model fine-tuning optimization procedures.
We demonstrate our approach by imposing specific compositional structures from a reference image onto generated outputs. We choose CIFAR-10 for our primary experiments as it provides an unbiased
7

---

SSIM Density: Patch PCA vs. Stable Diffusion
SSIM: Patch PCA vs, Stable Diffusion
1.0
Patch POAYs Statle Dimoucnn
Fandom Seed Paizs
00.018
0.8-
0.774
6.766
(875M))
0.40
0.406
Donsity
6-
5SIM
0.6-
4-
0.4
2-
0.0
0.2
0.6
0.8
0.0
SSIM
Number of PCA Components
(a) Original images
(b) PCA projections
Figure 5: SSIM score distributions comparing seed-paired candomly-pairee images. (a) Original images show higher similarity with identical noise seeds. (b) PCA projections show increased
similarity when projected onto lower-dimensional spaces.
Algorithm 1 Patch-based PCA Noise Editing with Frequency Band Control
I: Input: Original noise z, reference image x,, patch size p, number of channels c, eigenvectors {uy]z=1 interpolation factor a, frequency indices I 21, c. p²} Output: Edited noise
3: Extract overlapping x patches from z and x, to obtain (p. and {p.
4: for each patch index in parallel do
5:
Decompose in PCA basis: p poise Dep' cuiee uj and p!d Σグ C1 uy j=1
6: Split the noise patches p. paise into editing part and orthogonal part
pi ΣjEr
poss Uy
D. Part to be edited
8:
noise ontoog + -Ejez Gij
b. Preserved part
Store original norm of the editing part p. +- Sdit
10:
11:
// /dditin part coefficients
12:
vi <-
13:
JjEz Extract reference coefficients: c, 1e
Extract selected coefficients: ca [jE
14:
cniie eff
15:
Calculate angle between vectors: +- cos
T I
16:
Apply SLERP entire vector: Vi sin((1-o1d)
noise
sin( (off ein(O)
cfe
sin(O)
17:
Reconstruct edited part: vil < EjET Ve,y Uy where j' is the index of in
18:
// Normalize edited part to preserve original energy valt
19:
vedit p.dit.
20: Combine edited orthogonal parts
21: p' p.
orthing
edit
22:
end
23: Reconstruct z' by using the center pixel each patch p to reconstruct image
24:
Return z'
testing environment where no specific training guided editing has been performed, allowing to evaluate the intrinsic capabilities of diffusion models.
Methodology: Our patch-based noise editing approach (Algorithm
I consists of three steps:
Identifying specific frequency bands the patch PCA space that encode compositional information
2. Performing spherical interpolation (SLERP) between the noise and reference patches within these bands
3. Normalizing to preserve the energy of the original noise

---

Original
0.05
0.15 0.20 0.25 0.30 0.35 0.40
0.45 0.50
Figure 6: Progression from original image (leftmost, seed 3) to target reference image (rightmost) using zero-shot editing. Middle images generated with increasing interpolation factors (a from 0.05 to 0.5) using principal components range [1, 9]. Note the gradual emergence the circular composition from the target image while preserving the semantic content of the original.
(a) Generations without editing.
(b) Editing in [L, 9] range.
Editing in [1,9]] (2, 5) range.
Figure 7: Zero-shot image editing on Cifar-10 10. (a) Original images generated from seeds 0-7 without editing. (b) Images generated with noise modified in frequency band [1. 9] using a = 0.25. Note the circular composition but with colors influenced by the reference image. (c) Images generated with noise modified in frequency band [1, 9] \ f2, 5) using a 0.25. By excluding color-encoding eigenvectors, original colors are preserved while circular composition is more prominent.
We utilize the leading 9 eigenvectors from the patch PCA decomposition, which effectively capture essential compositional information in 5 patches. Using a pre-trained EDM model on CIFAR- 10, test approach with distinctive red target image (Figure 6 rightmost) reference exhibits both centered and circular compositional properties. We establish random noise initializations (seeds 0-3), with unedited generation results shown in Figure 7a
reference. This
Results and Analysis: Figure 7 demonstrates the effectiveness of our approach. When editing low- frequency components (Figure 7b the circular target structure is clearly imposed while maintaining datasct-specific characteristics. Our analysis revealed that eigenvectors 2 and 5 primarily encode color information rather than compositional structure. When these vectors excluded from the editing process (Figure 7c). the model preserves the original color tones while still adopting the target composition, highlighting the specificity of patch principal directions. To quantify compositional alignment, we report the SSIM between edited outputs and reference compositions. To assess color preservation independently,
compute the Bhattacharyya coefficient [5] (range 0 to 1, higher means more similar) between RGB histograms of the edited image and the original (no-edit) image. A higher Bhattacharyya coefficient indicates better color similarity. These results summarized in
Figure 8: Zero-shot image editing applied to ImageNet goldfish (class 1) using seeds 0-9 with interpolation factors a 0, 0. and eigenvectors [1, [1, 2, 5). Original images (top) compared to edited versions (middle and bottom). Edited images increasingly adopt the reference circular composition while maintaining semantic content. Note how some seeds (e.g., seed 6) creatively forms the circular pattern while preserving the goldfish identity across interpolation strengths.

---

Table 2
Together, these metrics confirm that low-frequeney editing improves structural alignment with the reference, and by excluding pure color channels eigenvectors, better preserve the color while maintaining structural control.
Table 2: CIFAR-10, 256 seeds. Low-frequency editing improves structure (SSIM f). Channel-awer bands better preserve color.
Metric
No Edit
Edit [1, 9]
Edit 9] \ (2,5)
SSIM vs Ref
0.0844 + 0.0708
0.2521 + 0.0665
0.2887 + 0.0812
Color Hist Corr vs Ref J.
0.0402 + 0.0860
0.2515 + 0.0827
0.1110 + 0.0987
Color Hist Corr vs No Edit
0.3809 0.1137
0.5759 + 0.1130
We extend our experiments ImageNet with class conditioning on "goldfish" (class 1) using the same target-shaped reference. Figure paeurentssssetsss uncurated results with seeds 0-9. The results show that our noise manipulation technique successfully guides the model to generate images with the desired circular target composition while maintaining class-specific semantic content. Interestingly, the model exhibits creative variations in implementing the centered circular composition, suggesting an inherent flexibility in how diffusion models interpret and apply compositional constraints. More editing results can be found Appendix AA4 in the Appendix with different reference images, incorporating masks for localized editing, and its editing effect in the latent space of Stable Diffusion.
Our findings demonstrate a di diffusion model's s ability to adapt to imposed compositional structures cues while adding dataset-specific details. This suggests that the generative capabilities of these models extend beyond simply reproducing training examples to intelligently combining imposed structural constraints with learned semantic features. The effectiveness of our approach across both CIFAR- 10 and ImageNet demonstrates the potential for zero-shot image editing through noise manipulation and the benefit of leveraging frequency bands identified by patch PCA decomposition. We include in the Appendix experiments of noise editing high-frequen band and object-count alignment settings (see Appendix A.4), which further demonstrate the broad effectiveness of our approach.
Limitations
While our approach effectively captures compositional cues in diffusion models, some limitations remain. The observed correspondence between pixel-space and latent diffusion model eigenvectors suggests interesting connections that warrant further investigation. A comprehensive evaluation of our training-free free zero-shot noise editing in complex multi-object scenes settings is beyond the scope of this work and is left for future exploration.
8 Acknowledgments
The authors thank anonymous reviewers for their constructive feedback. This material is based upon work supported by NSF (National Science Foundation) via grants SCF-2112665 DMS-2502083, and DMS-2502084, by the Office of Naval Research (ONR 1000142412631), Advanced Research Projects Agency (DARPA) under Contract No. 1250010. We also acknowledge the Delta HPC cluster at the NCSA (National Center for Supercomputing Applications), University of Illinois, accessed via the NSF ACCESS program (allocation no. TG-CIS220009).
well as by the Defense
10

---

References
[1] Michael S Albergo and Eric Vanden-Eijnden. Building normalizing flows with stochastic interpolants. arXiv preprint wwXiv:2209./557/, 2022.
[2] Monagi Alkinani and Mahmoud R El-Sakka. Patch-base models and algorithms for image denoising: comparative review between patch-based images denoising methods for additive noise reduction. EURASIP Journal on Image and Video Processing, 2017:1-27, 2017.
[3] Theodore Wilbur Anderson. An introduction to multivariate statistical analysis, volume 2.
Wiley New York, 1958.
[4] Fan Bao, Chongxuan Li, Yue Cao, and Jun Zhu. All are worth words: a vit backbone for score-based diffusion models. In NeurIPS 2022 Workshop on Score-Based Methods, 2022.
[5] Anil Bhattacharyya. On measure of divergence between two statistical populations defined by their probability distribution. Bulletin of the Calcutta Mathematical Society, 35:99-1100 1943.
[6] Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung- Woo Ha. Stargan v2: Diverse image synthesis for multiple domains. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 8188-8197, 2020.
[7] Charles-Alban Alban Deledalle, Joseph Salmon, Arnak S Dalalyan, et al. Image denoising with patch based pea: local versus global. In BMVC, volume 81, pages 425-455 2011.
[8] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. 2009 IEEE conference on computer vision and pattern recognition, pages 245-255. leee, 2009.
[9] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. Advances in neural information processing systems, 34:8780-8794 2021.
[10] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenbom, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition scale. arXiv preprint arXiv:2010. / /929, 2020.
[11] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Dominik Lorenz Yam Levi, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learning, 2024.
[12] Yuan Gao, Jian Huang, and Yuling Jiao. Gaussian interpolation flows. Journal of Machine Learning Research, 25(253):1-52, 2024.
[13] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances neural information processing systems, 33:6840-6851, 2020.
[14] Guillaume Huguet, James Vuckovic, Kilian Fatras, Eric Thibodeau-LLaufr, Pablo Lemos, Riashat Islam, Cheng-Hao Liu, Jarrid Rector-Brooks, Tara Akhound-Sadegl Michael Bronstein, et al. Sequence-augmented se (3)-flow matching for conditional protein backbone generation. arXiv preprint arXiv:2405.20311 2024.
[15] Mason Kamb and Surya Ganguli. An analytic theory of creativity in convolutional diffusion models. arXiv preprint arXiv2241..20292, 2024.
[16] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings the IEEE/CVF conference computer vision and pattern recognition, pages 4401-4410 2019.
[17] Tero Karras, Miik Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35: 26565-26577, 2022.
11

---

[18] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images, 2009.
[19] Shuangqi Li, Hieu Le, Jingyi Xu, and Mathieu Salzmann. Enhancing compositional text-to- image generation with reliable random seeds. arXiv preprint arXiv:2411.1881 2024.
[20] Xiang Li, Yixiang Dai, and Qing Qu Understanding generalizability of diffusion models requires rethinking the hidden gaussian structure. Advances in neural information processing systems, 37:57499-57538 2024.
[21] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, 2022.
[22] Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In International conference on learning representations (ICLR), 2023.
[23] Jiafeng Mao, Xucting Wang, and Kiyoharu Aizawa. The lottery ticket hypothesis in denoising: Towards remantic-driven initialization. In European Conference Computer Vision, pages 93-109. Springer, 2024.
[24] Matthew Niedoba, Berend Zwartsenberg, Kevin Murphy, and Frank Wood. Towards a mecha- nistic explanation of diffusion model generalization. arXiv preprint 1..132241..1933,, 2024.
[25] Karl Pearson. Liii on lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin philosophical magazine and journal science, 2(11):55-557, 1901.
[26] Robin Rombach, Andreas Blatmaann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695 2022.
[27] Jascha Sohl- Dickktein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. In International conference on machine learning, pages 2256-2265. PMLR, 2015.
[28] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021.
[29] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
[30] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.
[31] Alexander Tong, Kilian FATRAS, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector-Brooks Guy Wolf, and Yoshua Bengio. Improving and generalizing flow-based genera- tive models with minibatch optimal transport. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL http:p///oopaarrrrroooeoeoooetee Certification.
Expert
[32] Apoorv Vyas, Bowen Shi, Matthew Le, Andros Tjandra, Yi-Chiao Wu, Baishan Guo, Jiemin Zhang, Xinyue Zhang, Robert Adkins, William Ngan, et al. Audiobox: Unified audio generation with natural language prompts. arXiv preprint arXiv:2312. 15821, 2023.
[33] Zhengchao Wan, Qingsong Wang, Gal Mishne, and Yusu Wang. Elucidating flow matching ODE dynamics via data geometry and denoisers. Proceedings of the 42nd International Conference on Machine Learning, pages 62020-62083. PMLR, 2025.
[34] Binxu Wang and John Vastola. The unreasonable effectiveness of gaussian score approximation for diffusion models and its applications. Transactions Machine Learning Research.
12

---

[35] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Ecro P Simoncelli. Image quality assessment: from enror visibility to structural similarity. IEEE transactions on image processing. 13(4): 600-612, 2004.
[36] Katherine Xu, Lingzhi Zhang, and Jianbo Shi. Good seed makes good crop: Discovering secret seeds in text-to-image image diffusion models. 2025 IEEE/CVE Winter Conference on Applications of Computer Vision (WACV), pages 3024-3034. IEEE, 2025.
[37] Huijie Zhang. Jinfan Zhou, Yifu Lu, Minzhe Guo, Peng Wang, Liyue Shen, and Qing Qu. The emergence of reproducibility and consistency in diffusion models. In Forty-firsl International Conference on Machine Learning, 2024.
[38] Richard Zhang, Phillip Isola, Alexei Efros, Eli Shechtman, and Oliver Wang. The unrea- sonable effectiveness of deep features as perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586-595, 2018.
[39] Zikai Zhou, Shitong Shao, Lichen Bai, Zhiqiang Xu, Bo Han, and Zeke Xie. Golden noise for diffusion models: A learning framework. arXiv preprint arXiv:2411.09502, 2024.
[40] Daniel Zoran and Yair Weiss. Natural images, gaussian mixtures and dead leaves. Advances in Neural Information Processing Systems, 25, 2012.
13

---

A Appendix
A.1 Proof of Theorem 5.1
distinctive feature of natural images is their spatial coherence, where neighhoring pixels exhibit strong correlations. Modern neural architectures like CNNs and vision transformers [T0] leverage this property by processing images through local patches. We prove that when a denoiser is constrained operate compuly- each pixel's output based solely on its surrounding patch-and when patch statistics follow a Gaussian distribution (a reasonable approximation for small natural image patches 40) ) the optimal denoiser is precisely the Patch PCA denoiser defined in Equation 4
Below, we establish general result about optimal denoisers within the class of admissible local functions (Definition I.. extending previous work by [15] boundary-broken equivariant local" functions to accommodate arbitrary patch shapes (not necessarily square-shape) and possibly different shapes for each pixel.
Definition A.I (Admissible local Functions). We let D = {1,..... dd2 denote the pixel lattice and 2 1 the number of color channels. Let ..,.... Pe z be finite patch shapes, each containing the origin as its center.
Each pixel For pixel x, its assigned patch defined as wi = {(x + r) r (ls(w)), centered at æ. Here we assume that the patch assignment satisfies that wi C D for all E D.
D is assigned exactly one patch shape through shape- map S D
kj. {l....,k).
function f ddxxxe + Rdxdxe is called admissible local function the output fI) each pixel equals the output of local function fs(a) applied to the surrounding patch (z( of the image at pixel x. That is, is admissible local there exists, for each shape index ...... measurable map fy glglxe + Re such that for every pixel I D:
f())(x) fs(=)(w2(1)
where wa (1) is the patch of image I extracted at location according to shape {2
Theorem A.2. Let I E Th Dxe be random image drawn from distribution with B| 712 < co. For any a > O, we consider the noisy observation defined below
1.I.d. N(O,T), T), a a> 0;
Y = oZ,
For each patch shape 22, we let j denote the law of wx, (1) where Xj is uniformly chosen from S, = fz E D S(x) = j}. We call the patch distribution of the patch shape j. Then, we have that the unique minimizer of the following loss function
argmin, admissible local TE [[IS(Y) -1
can be explicitly determined as follows:
(i) For each shape j = 1,...,k, we let Q3 == (1) - o oZg where Zj is an independent Gaussian noise with the shape S2 Then, we define
(1) = (((xxx Q2 a], q R2,
(5)
where (w)o denotes the central pixel of patch w corresponding to the patch shape Ω.
(ii) Then, the optimal denoiser f is given by
f')) = (wo (Y)),
where fÿ is defined in equation
Proof. Throughout the proof we use E[-] for expectation and || l2 for the Frobenius norm on dxxxxx For patch write wi E R2 for its centre pixel (the entry whose spatial offset is the origin).
For an admissible local function ʃ let
R()) [ - 11
Σ zED
(u)
(()))
This means that Z, follows the standard Gaussian distribution 12 9,1
14

---

Grouping the sum by shape index gives
RU)
Σ
Σ
[[Ils
-
(6)
==1 rES,
where = (ar E D: S(x) = j}. Hence each local map fj influences only the j-th summand and can be optimized independently.
Fix shape index and choose pixel Xj uniformly from Xj. Set
Pj wx, (T), where Zy has i..d........................ (V(,,) 1) entries indexed by Sg.
Q2 = Pj + Zz.
Note that
R₂())= = Σ ((Il)(w),())) [x))1]]] = - - -3,ol]
SES,
where P3.0 is the center pixel of the patch Pj. Since |Sy| is a constant factor, minimizing R is equivalent to minimizing
Rf())) (()))(()) - P..ol"].
whose unique minimizer is the posterior mean
(g) = E(P3.0 - g]
R Rxxx
which is exactly equation (5). Plugging these local optima into equation gives global risk R(f*) S R(f) for every admissible local function f; if any fy differs from fj on set of positive
g-measure, the corresponding R (and hence R) is strictly larger, proving uniqueness of f*
We now state the rigorous version of Theorem S.I by assuming that the patch prior is Gaussian. Corollary A.3 (Closed-form when the patch prior is Gaussian). Keep the notation of the preceding theorem and assume that, for a fixed patch shape j. the patch prior multivariate normal
Tj = MM,, Ei),
M4j R"', Σ, E xny
ng = [2 c.
Then, the optimal local map Equation 5 is the following affine function
()) 13,0 + (2, (9 p3)]o,
E RR
(8)
where the subscript 0 denotes the central pixel of the patch corresponding to the shape S2
In particular, we consider the orthogonal decomposition of E, below
U, E RuyXn, orthogonal, A, = diag(A), Ajiny). Σ =
Denote the column eigenvectors by uj. and the eigenvalues by Aj.r. r = ,....,,yy Let (ujrr 18 be the part of ujr corresponding to the central pixel of the patch. Then, Equation
( can be rewritten follows:
13(4) Mj.0 + Σ
Agr + o² 49- M-j Uyr) (ugr)o
Ajr
for every patch g E R105/xx
Proof. Write P, ~ (M E,) and Qj P, + oZ, with Z, ~ N'O, In,) independent. The pair (P3s Q;) is jointly Gaussian with
E[P3] μjj E[Q,] Fý. Cov(P,, E,, Cov(Q,) EE o² Iny- For jointly Gaussian vectors, the conditional mean is affine (see for example [ Theorem 2.5.
[P, Q g) μ, (E, + 0² o²In, ) (g 14)
Then, restricting everything to the central pixel gives Equation 8
15

---

Remark A.4. Now, assumee that each pixel is assigned the same patch shape (e.g.. (2k+1) x (2k+ square centered at the pixel), with eircular padding applied handle boundary pixels. In this uniform case, we can set S(x:) 1 for all E D, and with the patch prior assumed to be Gaussian, Equation (9) implies that the denoiser for all pixels coincides exactly with the patch PCA denoiser defined in Equation ( This observation also provides intuition for neural network denoisers: when trained on approximately Gaussian patch distributions and constrained to operate locally, their behavior is close to this optimal Patch PCA denoiser.
A.2 Supplementary material for Section 4
Supplementary algorithm for noise perturbation perturbation in Algorithm 2
We provide the formal algorithm for noise
Algorithm 2 Patch-PCA Band Resampling for Noise Perturbation
1: Input: Initial noise x E 824 orthonormal Patch-PCA basis {u.] cutoff index n with S n S K,
where K = x p²
2: Output: Modified noise ă with resampled high- lighfrreueeccy components
3: Partition into disjoint patches ( ) of size p (we non-overlapping 4 x patches in experiments).
4: for each patch z; do
Decompose the patch into Patch-PCA basis:
K
uy,
di (z., u₁)
for j == n to do 7: Replace coefficient as,, with new i.i.d. Gaussian sample:
Onj Eigh
Eij (0,
8: end for
9: Reconstruct the modified patch:
K ai, Uj
2,
10: end for
11: Reassemble all patches to obtain the modified noise image 2
12: Note: Because [u.} is orthonormal and is i.i.d. Gaussian, each ozj remains Gaussian; therefore,
2 ~ I).
13: Return 8
Supplementary on the covariance matrix from the patches For small-sized patches, the covari- ance matrix is robust to the collection of the patches. We sample 10, 000 patches from ImageNet, FFHQ, and AFHQ datasets and extract patches of different sizes, and compute and compare the covariance matrix from the same-sized patches but from different datasets. The result is shown in Figure 9 The covariance matrix is robust the dataset used for sampling the patches.
Supplementary similarity index measure (SSIM) The SSIM is an alternative to the MSE for measuring the similarity between two images. It is definded in Wang et al. (35] follows:
(2 + C2)(2015 C2) SSIM(I, (N² + p3 + Ci)(o) + +03
(10)
where I and J are the two images, Pı and μu are the mean of the images, of and o3 are the variance of the images, ØI is the covariance between the two images, and C₁ and C2 are constants to avoid division by zero. The SSIM is value between 0 and 1, where 1 means that the two images are identical. The SSIM a better measure for the structure similarity between two images than the MSE, as it takes into account the luminance, contrast, and structure of the images.
Supplementary on similarity measure through LPIPS The LPIPS perceptual similarity
measure between two images defined in Zhang et al. [38) We include the LPIPS results for the
16

---

Covariance Correlation Between Image Datasets Across Patch Sizes
1.00
Correlation
0.95
0.90
0.85
IMAGENET FFHQ
IMAGENET AFHQ
FFHQ AFHQ
0.80
8
16 Patch Size
32
64
Figure 9: Covariance matrices with different patch sizes constructed from different datasets. For small patch sizes (5 16), the covariance matrices almost coincide with each other (more than 0.99 correlation).
pixel-space and latent-space models in Table with the same seed and random pairs for both pixel- space and latent-space models. The LPIPS results show a similar trend as SSIM: images generated from identical seeds are consistently closer than those from random pairs. The effect strongest for pixel-space models such as ImageNet and FFHQ. Comparing with SSIM, LPIPS shows a smaller gap between the same seed and random pairs in general, possibly because LPIPS takes into account the perceptual similarity, which has more emphasis on high frequency components. This matches central result: seed-dependent information concentrates in low-frequency components, low- frequency focused similarity metrics like SSIM show a larger effect than high-frequency focused metrics like LPIPS.
Table 3: LPIPS (L) comparison between paired (same-sced) and random pairs across pixel-space and latent- space models. Lower values indicate higher perceptual similarity. Results are averaged over 256 seeds.
Network Pair
LPIPS (Same Seed)
LPIPS (Random)
Pixel-space models
ImageNet FFHQ
0.454 0.069
0.538 = 0.064
ImageNet AFHQ
0.486 0.099
0.577 0.093
ImageNet vs Patch PCA
0.552 0.101
0.626 0.092
FFHQ vs AFHQ
0.371 0.093
0.449 + 0.089
FFHQ Patch PCA
0.542 + 0.048
0.581 + 0.044
AFHQ Patch PCA
0.508 0.067
0.560 = 0.063
Latent-space models
Stable Diffusion vs Patch PCA
(.8995 + 0.059
0.951 + 0.048
We include additional evidence on the consistency of the seed-dependent structural pattern across different samplers, architectures, and flow-matching variants.
Deterministic ODE Samplers (EDM, VE, VP) For fixed pretrained diffusion model and fixed noise seed z, we compare three common deterministic samplers (default polynomial scheduling in EDM [17]. variance exploding (VE) schedule [28), and velocity preserving (VP) schedule (30). As shown Table the generated images are nearly identical across samplers, with high SSIM and low MSE between outputs. This indicates that the noise-to-image mapping is robust to the sampler choice deterministic settings, and our findings are applicable all of them.
Network Architectures: U-Net vs Transformer (U-VIT) To test whether the phenomenon depends on convolutional priors, we compare U-Net-based EDM model and transformer- based U- U-ViT trained on ImageNet-64 using public pre-trained models. Using the same initial noise, observe
17

---

PC1
PC2
PC3
Eigenvalue Spectrum
10a
10a
PC
PC
PC
Eigenvalue
20-4
PC
PC
PC
10-4
Number of components
(a) Eigenvalue spectrum the covariance matrix 4r4 patches.
(b) Eigenvectors of the covariance matrix of patches.
Figure 10: Patch PCA spectrum and eigenvectors. The eigenvalues follow power law distribution, and the leading cigenvectors contain the illuiination/colo variations, and edge detectors.
Pocchel
Resoher PC
Eigenvalue Spectrum
Elgenvalue
10a
Number of com components
(a) Power spectrum latent space eigenvectors show- ing characteristic power-law decay.
(b) Decoded top latent eigenvectors from 5 X patches.
Figure 11: Analysis of latent space PCA: (a) Power spectrum showing characteristic power-law distribution typical of natural image statistics; (b) Visualization of decoded top latent eigenvectors from x patches, showing progression from illumination/color variations to edge detectors.
Table Deterministic samplers yield nearly identical outputs for the same seed.
Sampler Pair
SSIM
MSE
EDM VE
0.9902 0.0275
2.36 10 4 9.776
EDM
0.9821 = 0.0386
5.90 1.4
VP vs VE
0.9899 = 0.0153
20 444
18

---

strong structural alignment between generated images. Similar patterns persist when we compare the U-ViT model with our Patch-PCA denoiser.
Table 5: Structure similarity between U-Net and U-ViT architectures using identical seeds.
Model Pair
SSIM
MSE 4
U-ViT vs EDM (paired)
0.8197 0.0794
0.0065 + 0.0045 0.0542 0.0263
U-VIT vs Patch-PCA (paired)
0.4805 0.1265
The high structural alignment between U-ViT and EDM suggests that the phenomenon of seed- dependent structural pattern is widely applicable to different architectures.
Flow-Matching variants The flo-mmtcchnn (FM) model is framework that extends the diffusion model with ODE sampler [I 2T [22]. We train flow-matching model with the OT scheduling function (or rectified flow) on AFHQ dataset and compare it with the pretrained EDM model on AFHQ dataset. The results are shown in Table We observe that the flow-matching model generates images that are structurally aligned with the EDM model when using the same initial noise. This aligns with finding in Zhang et al. [37] that the same seed consistently generates similar images across different architectures when trained on the same dataset. Our paper additionally shows that the freeereuency components generated images from the same seed are aligned across different image datasets.
Table 6: Flow-matching (FM) vs EDM on AFHQ: same-seed generations remain structurally aligned.
Model Pair
SSIM Paired
SSIM Random t
MSE Paired +
MSE Random +
FM vs EDM
0.7840 0.10 0.5116 + 0.0862
0.0581 0.0454 0.0315 + 0.0367
0.0104 0.0068 0.0394 + 0.0149
0.1168 0.0447
FM vs PCA
0.1019 = 0.0270
A.3
Supplementary material for Section
5
Patch size ablation We test the effect of different patch sizes on the denoising performance using Equation (4) with different patch sizes. We generate 512 different initial noises and use the different patch sized PCA denoiser as denoiser in the DDIM reverse process Equation (I). We then compare between the generated images from the same initial noise and the different patch sized PCA denoisers in terms of cosine similarity, MSE, and SSIM. The results are shown in Figure 12 We observe that the generated images are visually similar while larger patch sizes produces smoother images as shown in Figure 12a The cosine similarity, MSE, and SSIM are also similar between the generated images. The quantitative results are shown in Figures 12b 12d These results on cosine similarity, MSE, and SSIM show that the PCA denoiser is robust to the patch size used in the denoising.
We also include more projections of the generated images from the latent space PCA denoiser and the Stable Diffusion model for comparison. The projections are shown in Figure 14
Pixel-to-Latent Subspace Alignment Let V = span { Vi..... v be the top-k PCA directions of 8p x 8p pixel patches. Let & be the encoder from Stable Diffusion; define the encoded subspace E(V)) = span(E(v))...., ((vx) Let W4 be the top-k PCA directions of p x p latent patches. We report the subspace alignment score align( (V&,,, W&)-the the mean cosine of principal angles in Ta- 7 Pixel-to-latent subspace alignment consistently high (mean cosine 0.93-0.95), showing that the encoder's dimensionality reduction preserves the leading patchwise PCA directions that our method utilizes.
A.4 Supplementary material for Section 6
In this section, we provide additional details in Section 6 regarding the zero-shot noise editing
precedure.
Localized editing (though mask) We apply the noise editing procedure to specific region of the image by applying mask to the noise. That is, we only edit the noise in the region defined by the
19

---

15
51
Similarity
Patch
1.000
0.397
0.983
6.350
1.000
0.907
Paltch
0.980
0.987
0.997
1,400
15
31
Patch Size
(a) Generated images from different patch sizes.
(b) Cosine similarity of the generated images.
Mean Squared Error (MSE) Confusion Matrix Patch Size vs Patchh Size
Patch
0.000
0.002
0.009
0.033
1.000
0.933
0.922
Size
0.009
0.005
0.000
0.002
0.882
0.933
1.000
0.978
0.011
0.008
0.002
0.000
0.889
0.922
0.575
1.000
1.5
31
Patch Size
Patch Size
(c) MSE of the generated images.
(d) SSIM of the generated images.
Figure 12: Ablation study on the effect of patch size on constructing the PCA denoiser. (a) Generated images from different patch sizes. (b) Cosine similarity of the generated images. (c) MSE of the generated images. (d) SSIM of the generated images. The results show that the PCA denoiser robust to the patch size used in the denoising. The generated images similar in terms of cosine similarity, MSE, SSIM and visually.
Table 7: Subspace alignment (mean cosine, f better) for p=5. First 10 directions and coarse grid.
k 2
3
4
5
7
8
9
10
Avg
Align t 0.9984 0.9042 0.9960 0.9969
0.8702 0.9926 0.9369
0.9410
0.9361
0.9380
0.9510
k
10
20
30
40
50
60
70
80
90
100
Ave
Align t 0.9380 0.8884 0.8739
0.9196 0.9468
0.9057
0.9409
0.9441
0.9530 0.9899 0.9300
20

---

Stable Diffusion
Patch PCA
Figure 13: Stable Diffusion generated image vs. Patch PCA denoised image when projected onto the top-2 subspace. While structurally similar, the Stable Diffusion image projection shows strong red-brownish color tone that is absent in the Patch PCA denoised image projection. This discrepancy likely stems from the second leading eigenvector (shown in Figure II having a light blue color tone that contrasts with the aod-brownshh tone.
(a) Original generated images
(b) Top-1 subspace projection
(c) Top-2 subspace projection
(d) Top-3 subspace projection
(e) Top-4 subspace projection
(I) Top-5 subspace projection
Figure 14: Complete visualization of generated images from Stable Diffussin (top row in each subfigure) and Patch PCA model (bottom row) across different subspace projections. Each column shows the same noise seed (0-3). As more principal components included in the projections (from top-1 to top-5), more detailed features are captured while maintaining the fundamental compositional structure identified in liver-dimensional projections.
mask. An example of the mask is shown in Figure 15a where the red area is the area to be edited and the blue area is the area to preserved. We apply the same editing procedure in the experiment in Figure 7 but only to the area defined by the mask. The results are shown in Figure [6] We observe that the localized editing resulted in a more localized composition constraint. Also, similar to the previous experiment, when we do not include the eigenvectors corresponding to the color variations in our editing procedure, the generated image Figure [ is more similar reference image Figure T6a overall and only the top-right corner composition showing curved shape which aligns with the masked part of the target reference image Figure 15
21

---

O
(a) Top right comner mask. The red area is edited, the blue area preserved.
(b) Target reference image with central and surround- ing compositional structure.
Figure 15: Reference images used in our editing procedure. Left: The mask defines which areas of the noise to modify. Right: The target reference image whose compositional structure we aim to transfer.
(a) Reference generation without editing.
(b) Editing in range [1, 9] with mask applied.
(c) Editing in range [1, (2,5) with mask.
Figure 16: Editing results on CIFAR-10 with target reference using interpolation factor a 0.25 as in the main experiments. The mask is applied to restrict editing to only the top-right corner of the image. The resulting editing is shown in Figure 16b] and Figure T6c The reference generation is shown in Figure [6a| The edited images closely align with the reference image/compositional structure in the top-right corner.
More examples of reference images demonstrate our editing approach with a more complex compositional structure, we utilize Vincent van Gogh's 's iconic "Starry Night" painting as a reference image (shown in Figure| I7 resized to 32 32). This reference image contains strong diagonal movement from bottom left to middle right. We still use the same patch size of 4 x as in the experiment the main text and we choose the range [1, (1, 2, 5) to edit the noise with interpolation factor a ranging from 0.0 to 1.0 (a 0.0 is the original (non-edited) generation). The generated images shown in Figure 18 with initial noise seeds from 0 to 9. The generated images show a smooth transition from the original image to those with diagonal movement, like the reference Starry Night image. This shows that editing procedure can be used to transfer the compositional structure of a complex reference image to the generated images.
Figure 17: Ref- erence image used for inter- polation in Fig- ure 18
Examples of editing in latent diffusion model We also apply our noise editing procedure to latent diffusion models. We use patch size x 5 with the eigenvalue spectrum shown in Figure II and eigenvectors visualized in Figure lTb] We use an empty prompt and the simple target-shaped image tright-hand side of Figure 6) as the reference image.
First, we compare the effects including versus excluding color-focused eigenvectors our editing procedure. To demonstrate the versatility and control offered by our approach, we explore a full range of interpolation factors a from 0.0 to with 0.1 increments. Figure shows this comparison with two frequency ranges: [1, 12), which includes the top 4 color-focused eigenvectors (left), and [5, 12),
22

---

Seeds:
Figure 18: Generated images using the painting "Starry Night" by Vincent Gogh the reference image. The interpolation factor a ranges from 0.0 to 1.0 (a 0.0 is the original (non-edited) generation). The generated images show smooth transition from the original image toward variants with diagonal movement, similar to the reference Starry Night image.
23

---

Seed Seed 1
Seed 2 Seed 3
Seed 0
Seed
Seed
Seed
0.00
0.00
0.10
0.10
0.20
a= 0.20
d= 0.30
0= 0.30
a= 0.40
0.40
a= 0.50
0.50
Q= 0.60
a= 0.60
== 0.70
a= 0.70
0.80
0.80
== 0.90
== 0,90
a= 1.00
a 1.00
(a) Editing with range [5, 12] (excluding color-focused eigenvectors).
(b) Editing with range [1, 12] (including color-focused
eigenvectors).
Figure 19: Zero-shot noise editing latent diffusion models comparing two different frequency ranges with interpolation factor ranging from 0.0 to 1.0. Left: Using range [1, 12] which includes color-focused eigenvectors, resulting in both structural and color transfer. Right: Using restricted range [5, 12] which excludes color-focused eigenvectors, preserving original colors while adopting the target's 's compositional structure.
24

---

Seed Seed 1
Seed 2 Seed 3
Seed 0 Seed 1 Seed
Seed
0.00
0.00
0.05
0.05
0.10
a= 0,10
5003
a= 0.15
0= 0.15
0.20
0.20
0.25
0.25
a= 0.30
a 0.30
0.35
a= 0.35
0.40
0.40
== 0.45
== 0.45
a= 0.50
a= 0.50
(a) Editing with range [5. 12] (excluding color-focused eigenvectors).
(b) Editing with range [1, 12] (including color-focused eigenvectors).
Figure 20: fine-grained progression of zero-shot noise editing in latent diffusion model with inter- polation factors from 0.0 to 0.5 with 0.05 increments. Left: Using range [1, 12] which includes color-focused eigenvectors, resulting in both structural and color transfer. Right: Using range [5, 12] which excludes color-focused eigenvectors, preserving original colors while still adopting centered and circular compositional structure.
25

---

which excludes them (right). When color-focused eigenvectors are included, while the generated images align with the composition of the target-shaped reference image. they also exhibit strong resemblance to the color of the reference image; see Figure 19b] When these eigenvectors are excluded, the generated images maintain their original colors while still adopting the circular and centered compositional structure of the target, see Figure This demonstrates that our approach of choosing the frequency range effective in controlling the compositional structure and color transfer in the generated images.
To further investigate the gradual evolution of compositional structure, Figure 20 shows a more fine-grained progression of editing strength. With increasing values, we observe the composition of the generated images gradually shifting toward increasingly pronounced central and circular structures, even though the overall image does not change much, especially when the color-focused eigenvectors are excluded (Figure 20a
Noise editing in high-frequency bands. While our main experiments focus on low-frequency components for composition control, our Patch-PCA based noise editing method naturally extends to high-frequency bands. We test this by using Starry Night as the reference image (Figure [7] and 5 Patch-PCA basis. The reference image contains blue-yellow oil-painting texture and curly stylistic texture. We observe that by editing the high-frequency coefficients, we can achieve a texture transfer effect. This effect can be sensitive to the choice of frequency band to determine what type of texture to transfer. A visual example is shown in Figure 21] When we use the frequency band [19, 75] and with small interpolation factor of 0.3, we already see the blue-yellow oil-painting texture in the generated images. When we use the frequency band [20, 75). the blue-yellow oil-painting texture not as easy to generate, and we mostly pick up the curly stylistic texture even with large interpolation factor of 0.9. In all cases, high- high-frequency band noise editing leaves object layout largely unchanged while altering texture, complementing the cowffreuueccy compositional control. A more comprehensive study across the frequency bands is left for future work.
No edit
Edit [19:75]
Edit [20:75]
Noise editing in figh-freuuency bands results using frequency band [19, 75] and [20, 75] with interpolation factor 0.3 and a 0.9 respectively.
Figure 21:
Editing for Object Count (Three-Object Composition). To test whether our noise editing method can affect object count, we use the ImageNet class "granny smith apple" (class 948) and create reference image containing three circular white blobs on black background. We design the task for achieving three apples in the generated images. We edit the initial noise in the same way as the ImageNet editing experiment in Figure i.c., using low-frequency Patch-PCA bands [1, 9] \ {1, 5).
Results. Before editing, most samples contain either only one apple or more than three apples. After editing, 43/64 samples exhibit the intended three apples near the highlighted areas in our reference image, confirming that low-frequency structure can influence object counting (see Figure 23 However, we also observe that the background diversity almost removed. We leave it as future work to balance the object count and the background diversity.
Figure 22: Three-object reference used for count alignment.
26

---

Table 8: Distribution of object counts before and after low-frequency editing (64 seeds).
# Apples
0
1
2
4+
No Edit Edited
5
30
16
2
8
11
43
0
(a) No Edit (64 seeds).
(b) Edited (bands [1, 11 \ a=0.22).
Figure Object-count control via low-frequency editing. Left: unedited generations, most with one apple more than three apples. Right: edited generations, most with three apples in the area of the of circular white blobs of the reference image.
A.5 Datasets and pre-trained models
The datasets used in our experiments are ImageNet [8] FFHQ 16, AFHQ 66. and CIFAR-10 18
The ImageNet dataset contains 1.28 million images with 1,000 classes. The FFHQ dataset contains 70,000 high-quality images of human faces. The AFHQ dataset contains 15,000 high-quality images of animals (cats, dogs, and wild animals). The CIFAR- dataset contains 60,000 images 10 classes, with 6,000 images per class.
The pre-trained models used in our experiments are the diffusion models trained on ImageNet. FFHQ, and AFHQ datasets are available from the official repository of Karras et al. [17] https //github. com/NVIIIi Com/NVlabb /ww.bs/lbbbbbbb/eaaa/ edm and the transformer-based U- ViT model available from the official repository of Bao et al. [4] https::///// // thub. cwwcooooaaaaaaeee
Sann//aofff/U-/ We also train flow matching on the AFHQ dataset using repository of Tong et al. 31
https: //gi tigu..u..... com/ com/attng01/
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes],
Justification: We provide theoretical, methodological, and experimental analysis in our paper to support our claims in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
27

---

The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. It is fine to include aspirational goals as motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitation in the last section of our paper.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: We provide proof in the supplemental material for theoretical results in the main paper.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch provide intuition.
28

---

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide the setup and details of our experiments in the main paper and appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to hosted model (e.g., in the of a large language model), releasing of a model checkpoint, or other means that are appropriate the research performed.
While NeurlPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provide the algorithm in the paper which can be used to reproduce the results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cd for more details.
29

---

reproducible, they While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https www.ggiddcecoooo 7 unips.cc/publle httpa////c/...dsd///deeuuuiiieee Stwp://mseeeeeeisiiiiiieieaaiaie www.iisssiiniiiininon for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We provide the experimental setup and details in the main paper and appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification:
We provide error bars our experiments.
Guidelines:
The answer NA means that the paper does include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma emror bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
30

---

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The experiments our paper does not require much compute resources and the provided information is sufficient to reproduce the experiments.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
We have reviewed the NeurIPS Code of Ethics and our research conforms to it.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our work aim to understand the diffusion model and not directly related to any societal impact.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinfornation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
31

---

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer[[NA]
Justification:
This paper does not release any data or models.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We cite the original paper that produced the code package or dataset.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website). the copyright and terms of service of that source should provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, saperswwthcode cwm/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
This paper does not release new assets.
32

---

Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation,
or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer[[NA]
Justification:
This paper does not involve human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: This paper does involve LLMs as core component of the methodology.
Guidelines:
33

---

The answer NA means that the core method development in this research does not involve LLMs as any important. original, non non-standard components.
Please refer to our LLM policy /https://neurig: /ttp://pp.ccc//offfeeeee///////s for what should or should not be described.
34


Reviews:
[{"review_id": "rUn9Y5YOih", "reviewer": "Reviewer_t5kH", "summary": "This paper explores the influence of initial noise seeds in diffusion models, focusing on how their low-frequency components encode universal compositional cues that govern the global structure of generated images. Through a detailed Patch-wise Principal Component Analysis (Patch PCA), the authors demonstrate that these structural signals persist consistently across different datasets and architectures, suggesting they are inherent properties of the generative process itself rather than artifacts of specific models or data.\nBuilding on this insight, the authors propose a simple, interpretable, and training-free Patch PCA denoiser, which approximates the structural layout of generated images using only statistical properties of natural image patches. This denoiser is theoretically grounded under a local Gaussian patch distribution assumption and further validated through extensive experiments. Moreover, the paper introduces a zero-shot noise editing technique, which allows controllable manipulation of image composition by interpolating noise components within specific PCA frequency bands without model fine-tuning or additional training.", "questions": "I don't have other questions. Please refer to the weakness above.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n●\tThe paper presents a compelling observation that the compositional structure of diffusion-generated images is predominantly determined by the low-frequency components of the initial noise. This is thoroughly validated via frequency band perturbation analysis, showing that perturbing low-frequency components significantly alters image structure, while high-frequency perturbations have negligible effect.\n\n●\tThe compositional cues encoded in noise seeds are shown to generalize across architectures (e.g., pixel-based and latent diffusion models) and training datasets. This cross-domain consistency suggests a deeper universality in the noise-to-image mapping of diffusion models.\n\n●\tThe idea of leveraging these structural cues for zero-shot image editing is novel and practical. The proposed editing approach operates solely on the noise level and maintains semantic coherence and structural layout using minimal assumptions. Qualitative examples show clear compositional transitions while preserving content.\n\n●\tThe paper includes extensive visualizations, ablations, and quantitative analyses that strongly support the authors’ observation. The method is well-motivated both empirically and theoretically, and presented with clarity.\n\nWeaknesses\n\n●\tWhile the Patch PCA denoiser is theoretically justified in pixel space under Gaussian patch assumptions, its extension to latent diffusion models is only evaluated empirically. Since the latent space is itself a learned low-dimensional representation, further theoretical or statistical analysis of PCA behavior in latent space would strengthen the argument for universality.\n\n●\tThe core theoretical justification relies on the assumption that image patches follow a Gaussian distribution. Whether this assumption holds in latent space for latent diffusion models is unclear and not discussed. \n\n●\tAlthough the paper provides promising qualitative results for the zero-shot editing method, it lacks quantitative metrics (e.g., compositional similarity scores, human preference, or alignment accuracy) to objectively assess the success of compositional transfer. Moreover, claims that both structure and color can be selectively preserved or transferred through PCA manipulation are not fully substantiated by quantitative evidence.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thoughtful and constructive feedback. We are encouraged that you found our central observations compelling, particularly the role of low-frequency components of initial noise in shaping compositional structure and the universality of this phenomenon across architectures and datasets. Below, we address the questions raised:\n\n----\n\n### 1. **Patch PCA in Latent Diffusion Models**\n>While the Patch PCA denoiser is theoretically justified in pixel space under Gaussian patch assumptions, its extension to latent diffusion models is only evaluated empirically. Since the latent space is itself a learned low-dimensional representation, further theoretical or statistical analysis of PCA behavior in latent space would strengthen the argument for universality.\n\n>The core theoretical justification relies on the assumption that image patches follow a Gaussian distribution. Whether this assumption holds in latent space for latent diffusion models is unclear and not discussed.\n\nWe appreciate the reviewer's questions regarding Patch PCA in the latent space. Due to the nonlinearity of the VQ-VAE encoder, a Gaussian patch distribution may no longer remain statistically Gaussian in latent space. Extended theoretical analysis is an interesting direction, but it can be quite complicated. However, we observe that the encoder's dimensional reduction aligns remarkably well with PCA for **image patches**: the leading principal directions in pixel space strongly correspond to the principal directions in latent space—precisely the components our PCA-based editing method leverages.\n\nTo support this, we conducted a subspace similarity analysis:\n\n* Step 1: Extracted 8p × 8p patches from natural images and computed the top k PCA directions in pixel space, where k ranges from 1 to 4p² (the dimensionality of the encoded latent patches).\n* Step 2: Encoded these PCA directions with the LDM encoder, yielding k vectors in latent space.\n* Step 3: Computed PCA directions directly on p × p latent patches from encoded images.\n* Step 4: Compared the two k-dimensional subspaces via mean cosine similarities of principal angles. The score ranges from 0 (orthogonal) to 1 (identical).\n\nBelow are the results for 5×5 latent patches used in the paper:\n\n**First ten principal directions:**\n\n| k | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | Average |\n|-----|---|---|---|---|---|---|---|---|---|----|---------| \n| **Subspace Alignment ↑** | 0.9984 | 0.9042 | 0.9960 | 0.9969 | 0.8702 | 0.9926 | 0.9369 | 0.9410 | 0.9361 | 0.9380 | 0.9510 |\n\n**Coarser grid up to full latent-patch dimensionality (100):**\n\n| k | 10 | 20 | 30 | 40 | 50 | 60 | 70 | 80 | 90 | 100 | Average |\n|-----|----|----|----|----|----|----|----|----|----|----|---------|\n| **Subspace Alignment ↑** | 0.9380 | 0.8884 | 0.8739 | 0.9196 | 0.9468 | 0.9057 | 0.9409 | 0.9441 | 0.9530 | 0.9899 | 0.9300 |\n\nThese results confirm our hypothesis with consistently high alignment values (>0.93 on average), supporting the validity of our patch PCA-based observations and methods in latent space. We will include this analysis in the revision.\n\n### 2. **Quantitative Evaluation of Zero-Shot Editing**\n\n>Although the paper provides promising qualitative results for the zero-shot editing method, it lacks quantitative metrics (e.g., compositional similarity scores, human preference, or alignment accuracy) to objectively assess the success of compositional transfer. Moreover, claims that both structure and color can be selectively preserved or transferred through PCA manipulation are not fully substantiated by quantitative evidence.\n\nWe agree that adding quantitative evaluation would further strengthen the experiments. To complement the qualitative results in Figure 7, we added quantitative evaluations across 256 random seeds using CIFAR-10. We use SSIM for structure alignment and Color Histogram Correlation for global color similarity. Color Histogram Correlation compares normalized RGB histograms using Bhattacharyya distance (range: 0 to 1, where 1 indicates identical distributions).\n\n| Metric | (a) No Edit | (b) Edit [1,9] | (c) Edit [1,9] \\ {2,5} |\n|--------|-------------|----------------|------------------------|\n| SSIM vs. Ref ↑ | 0.0844 ± 0.0708 | 0.2521 ± 0.0665 | 0.2887 ± 0.0812 |\n| Color Hist. Corr. vs. Ref ↓ | 0.0402 ± 0.0860 | 0.2515 ± 0.0827 | 0.1110 ± 0.0987 |\n| Color Hist. Corr. vs. No Edit ↑ | N/A | 0.3809 ± 0.1137 | 0.5759 ± 0.1130 |\n\nResults show that editing low-frequency Patch PCA components significantly improves SSIM, confirming better compositional alignment. Excluding color channels (c) achieves better color preservation, as demonstrated by higher correlation with the no-edit case and lower correlation with the reference image. We will include these quantitative results in the revision."}, {"review_id": "Q6FbFxoQ06", "reviewer": "Reviewer_HRUH", "summary": "The paper presents an empirical study on the latent noise variables used to seed image-diffusion model generations. The authors use patch-PCA to find that lower frequency components of the noisy latent can strongly influence the compositional structure and semantics of the image while higher frequency components have a smaller impact instead affecting attributes like texture. The authors then propose a zero-shot image editing technique using the PCA frequency components that enables style transfer.", "questions": "[L114] Can you please describe line 114 in more detail? i.e. “resampling the identified frequency bands [n,48] on each patch.” An algorithm like algorithm 1 would be nice (can be included in the appendix).  \n[Fig 1b] what do the three rows correspond to?  \n[L132] Are different models are being used here? Are you using an ImageNet trained model A and a FFHQ trained model B and then generating images with the same seed?  \n[Tab 1, Fig 5] How do the differences measure in terms of the LPIPS metric?  \nWas a patch size of 4x4 used for all experiments? How do results from the noise editing algorithm differ when using different patch sizes?", "limitations": "The authors do discuss limitations at the end of the main text, which I think fairly summarizes the shortcomings of this study.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:  \n**(S1)**: The paper does a thorough and interesting investigation into the compositional patterns of noise for diffusion models. The findings are non-trivial and relevant to the broader field as they show that diffusion models learn compositional features and data structure across different “frequencies” and that this is highly correlated with the initial noisy latent across datasets and models.  \n**(S2)**: The study generalizes to latent-diffusion models which are more widely used.   \n**(S3)**: The paper proposes a new zero-shot editing technique that can be used for image or style translation tasks based on the empirical findings with their patch-pca investigation.  \n\nWeaknesses:  \n**(W1)**: Zero-shot patch pca based noise editing is missing quantitative comparisons. I’m very curious about how this compares to methods like SDEdit, and if some of the insights of this technique can apply to SDEdit or vice-versa. It would be nice to show comparison with other zero-shot image and style editing techniques as well.  \n**(W2)**: How does the choice of sampler affect the findings of the paper? If I’m not mistaken, the study is currently using DDIM. Does this change for DDPM (given the stochasticity of each denoising step) or other VE/VP samplers?  \n**(W3)**: How would this analysis change for flow-matching based models? I don’t see a discussion of this, but I’m curious to know if similar patterns emerge.  \n\nOverall, this paper is very interesting and presents findings that should be relevant to the broader diffusion community. It is clearly written and the investigation seems sound. I would recommend it for acceptance.", "was_revised": true, "final_justification": "I maintain my recommendation for acceptance.\n\nThis is an interesting research paper with non-trivial findings. My reasons are outlined in the strengths section, and most of my concerns have been addressed in the rebuttal. I think this work's findings should be shared with the NeurIPS community.", "rebuttal": "We thank the reviewer for the constructive feedback and are encouraged by your recognition of the non-trivial nature of our findings and the practical value of the proposed zero-shot editing method. We address the questions below.\n\n### **1. Quantitative Evaluation of Zero-Shot Editing**\n>Zero-shot patch pca based noise editing is missing quantitative comparisons.\n\nWe agree that adding quantitative evaluation would further strengthen the experiments. To complement the qualitative results in Figure 7, we added quantitative evaluations across 256 random seeds using CIFAR-10. We use SSIM for structure alignment and Color Histogram Correlation for global color similarity. Color Histogram Correlation compares normalized RGB histograms using Bhattacharyya distance (range: 0 to 1, where 1 indicates identical distributions).\n\n| Metric | (a) No Edit | (b) Edit [1,9] | (c) Edit [1,9] \\ {2,5} |\n|--------|-------------|----------------|------------------------|\n| SSIM vs. Ref ↑ | 0.0844 ± 0.0708 | 0.2521 ± 0.0665 | 0.2887 ± 0.0812 |\n| Color Hist. Corr. vs. Ref ↓ | 0.0402 ± 0.0860 | 0.2515 ± 0.0827 | 0.1110 ± 0.0987 |\n| Color Hist. Corr. vs. No Edit ↑ | N/A | 0.3809 ± 0.1137 | 0.5759 ± 0.1130 |\n\nResults show that editing low-frequency Patch PCA components significantly improves SSIM, confirming better compositional alignment. Excluding color channels (c) achieves better color preservation, as demonstrated by higher correlation with the no-edit case and lower correlation with the reference image. We will add these results to the main text.\n\n### **2. Relation to SDEdit and Other Zero-Shot Editors**\n>I'm very curious about how this compares to methods like SDEdit, and if some of the insights of this technique can apply to SDEdit or vice-versa. It would be nice to show comparison with other zero-shot image and style editing techniques as well.\n\nSDEdit interpolates a reference image with noise at selected timesteps, then denoises from that intermediate state. Our method manipulates Patch PCA frequency bands in the initial noise. These approaches introduce editing at different aspects: SDEdit at selected timesteps, ours via frequency bands. Combining our approach with SDEdit for finer-grained control is an interesting direction. Unfortunately, this year's policy does not allow us to present visual results, and we will leave it as future work. We agree that a full benchmark across major zero-shot editors would be valuable. However, it falls outside this paper's scope, as our primary goal is to expose the structural role of low-frequency noise and introduce a novel, lightweight, training-free editing tool. We will leave comprehensive benchmarking as future work.\n\n### **3. Sampler Effect**\n>How does the choice of sampler affect the findings of the paper? If I'm not mistaken, the study is currently using DDIM. Does this change for DDPM (given the stochasticity of each denoising step) or other VE/VP samplers?\n\nYes, our study uses DDIM (EDM scheduling). Other deterministic samplers (VE, VP) produce identical results for the same model and initial noise when given sufficient steps. Our observations and methods apply to all deterministic samplers, empirically verified by near-identical images across samplers:\n\n| Sampler Pair | SSIM | MSE |\n|--------------|------|-----|\n| EDM vs VE | 0.990235 ± 0.027491 | 0.000236 ± 0.000976 |\n| EDM vs VP | 0.982148 ± 0.038618 | 0.000590 ± 0.001448 |\n| VP vs VE | 0.989944 ± 0.015296 | 0.000224 ± 0.000444 |\n\nHowever, stochastic samplers like DDPM add noise at every step, so the same seed no longer yields unique output. Our observations about low-frequency components and our editing method don't apply to stochastic samplers. One workaround is to extend frequency-band editing to intermediate denoising steps, which fall into the combination of our method with SDEdit, as you mentioned in the review. \n\n### **4. Flow-Matching Based Models**\n>How would this analysis change for flow-matching based models? I don't see a discussion of this, but I'm curious to know if similar patterns emerge.\n\nFlow-matching models extend diffusion models with deterministic sampling. We trained a flow-matching (FM) model with the OT scheduling function (or rectified flow) on AFHQ and compared it with pretrained EDM:\n\n| Network Pair | SSIM Paired | SSIM Random | MSE Paired | MSE Random |\n|--------------|-------------|-------------|------------|------------|\n| FM AFHQ vs EDM AFHQ | 0.7840 ± 0.1023 | 0.0581 ± 0.0454 | 0.0104 ± 0.0068 | 0.1168 ± 0.0447 |\n| FM AFHQ vs Patch PCA | 0.5116 ± 0.0862 | 0.0315 ± 0.0367 | 0.0394 ± 0.0149 | 0.1019 ± 0.0270 |\n\nThis suggests flow-matching models (FM AFHQ) learn similar noise-to-image mappings as EDM AFHQ. This aligns with existing studies on noise-to-image map reproducibility across different architectures. For example, the work [24] shows different architectures (U-Nets, U-ViT, Consistency Trajectory Models) produce remarkably similar images given identical initial noise in their Fig. 1 and Fig. 11. Consequently, our observations and methods are directly applicable to flow-matching models. We will add these discussions to the main text.\n\n### **5. [L114] Resampling the Frequency Band [n, 48]**\n>[L114] Can you please describe line 114 in more detail? i.e. \"resampling the identified frequency bands [n,48] on each patch.\" An algorithm like algorithm 1 would be nice (can be included in the appendix).\n\nWe decompose initial noise into 48 orthonormal Patch-PCA eigenvectors (3 channels × 4×4). To \"resample\" band [n,48], we set coefficients in range [n,48] to new i.i.d. Gaussian samples. Since the basis is orthonormal, overall noise remains Gaussian. We are happy to present this as an algorithm in the revision.\n\n### **6. [Fig 1b] Row Description**\n>[Fig 1b] what do the three rows correspond to?\n\nEach set of three rows shows three independent examples (top to bottom) of images generated after perturbing initial noise in the specified frequency band (labeled above the first row).\n\n### **7. [L132] Models Used**\n>[L132] Are different models are being used here? Are you using an ImageNet trained model A and a FFHQ trained model B and then generating images with the same seed?\n\nYes. We use pixel-diffusion models separately trained on ImageNet, FFHQ, and AFHQ. For each model, we generate images with identical seeds to test whether similar global structure emerges across models trained on different datasets.\n\n### **8. [Tab 1, Fig 5] LPIPS Metric Results**\n>[Tab 1, Fig 5] How do the differences measure in terms of the LPIPS metric?\n\nLPIPS results mirror SSIM trends. Same-seed images always receive lower scores (higher similarity) than random pairs. Differences are most pronounced in pixel-space models and smaller but evident for Patch PCA in latent space.\n\n**Pixel-space models:**\n| Network Pair | LPIPS (↓) Paired | LPIPS (↓) Random |\n|--------------|------------------|------------------|\n| ImageNet vs FFHQ | 0.454 ± 0.069 | 0.538 ± 0.064 |\n| ImageNet vs AFHQ | 0.486 ± 0.099 | 0.577 ± 0.093 |\n| ImageNet vs Patch PCA | 0.552 ± 0.101 | 0.626 ± 0.092 |\n| FFHQ vs AFHQ | 0.371 ± 0.093 | 0.449 ± 0.089 |\n| FFHQ vs Patch PCA | 0.542 ± 0.048 | 0.584 ± 0.044 |\n| AFHQ vs Patch PCA | 0.508 ± 0.067 | 0.560 ± 0.063 |\n\n**Latent-space models:**\n| Network Pair | LPIPS (↓) Paired | LPIPS (↓) Random |\n|--------------|------------------|------------------|\n| Stable Diffusion vs Patch PCA | 0.895 ± 0.059 | 0.951 ± 0.048 |\n\n### **9. Patch-Size Sensitivity**\n>Was a patch size of 4x4 used for all experiments? How do results from the noise editing algorithm differ when using different patch sizes?\n\nFigure 1's frequency perturbation used 4×4 patches to divide 64×64 (or 32×32) images, preserving Gaussian properties. Image editing experiments used 5×5 patches for both pixel and latent space. We tested 3×3, 7×7, and 9×9 patches with similar qualitative results, though optimal eigenvector numbers and interpolation weights varied. Larger patches generally require more eigenvectors for comparable results.\n\n---\n\n[24] Huijie Zhang, Jinfan Zhou, Yifu Lu, Minzhe Guo, Peng Wang, Liyue Shen, and Qing Qu. The emergence of reproducibility and consistency in diffusion models. ICML 2024."}, {"review_id": "kGEmcxj6mY", "reviewer": "Reviewer_azVL", "summary": "This paper investigates how initial noise structure influences the final output of diffusion models. Using patch-wise Principal Component Analysis (PCA), the authors empirically show that low-frequency components of the noise seed predominantly determine the compositional structure of the generated image. Based on this insight, the paper demonstrates that these compositional cues are universal across different models and datasets and proposes a simple Patch PCA denoiser and a zero-shot editing method to control image composition.", "questions": "I would like to see more theoretical analysis, and more exploration on high-frequency noise.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The paper provides a strong and intuitive investigation into the noise-to-image map of diffusion models, offering a clear link between low-frequency noise and image composition. This provides valuable insight into the underlying mechanics of the generative process.\n2. A key strength is the compelling empirical evidence that identical noise seeds produce structurally similar images across different models and datasets. This finding of \"universal compositional cues\" is significant and well-supported by both quantitative (SSIM/MSE) and qualitative results.\n3. The proposed zero-shot editing method is a practical and valuable application of the paper's findings. It offers an efficient way to guide image composition without needing any model retraining or fine-tuning, which is a significant advantage over many existing editing techniques.\n\nWeaknesses:\n1. The analysis focuses heavily on the role of low-frequency components for composition, while the influence of high-frequency noise components on texture and fine details is mentioned but not explored in depth. A more complete picture would involve a deeper analysis of how different frequency bands contribute to various aspects of the final image.\n2. The theoretical justification relies on a simplified Patch PCA denoiser, which, while effective, may not fully capture the complex, non-linear dynamics of large-scale neural network denoisers. This simplification could limit the direct applicability of the theoretical claims to state-of-the-art diffusion models.\n3. While the editing results on CIFAR-10 and ImageNet are promising, the method's effectiveness on more complex scenes with multiple objects or intricate layouts is not fully demonstrated. The compositional control appears to be primarily global, and its precision in more complex scenarios remains an open question.", "was_revised": true, "final_justification": "Thanks to the author for the response. Some of my questions have been resolved. Therefore, I will maintain my original score.", "rebuttal": "We thank the reviewer for the encouraging assessment and for highlighting both the empirical insight of universal compositional cues and the practical value of our zero-shot editing method. Below, we respond to each weakness and question.\n\n### **1. The Influence of High-Frequency Noise**\n>The analysis focuses heavily on the role of low-frequency components for composition, while the influence of high-frequency noise components on texture and fine details is mentioned but not explored in depth. A more complete picture would involve a deeper analysis of how different frequency bands contribute to various aspects of the final image.\n\nWe agree that a comprehensive exploration of all frequency bands would be valuable. However, in this paper, our focus is on establishing the dominant structural role of low frequencies. To demonstrate that the editing method naturally extends to the high-frequency region, we conducted preliminary experiments on high-frequency perturbation. Using \"Starry Night\" as the reference image (Figure 17, appendix) with 5×5 Patch PCA basis, we find:\n\n1. **Frequency band [19:75], interpolation factor 0.3**: Creates blue and yellow oil-painting texture like in Starry Night while maintaining the original composition.\n2. **Frequency band [20:75], interpolation factor 0.9**: Generated image remains almost identical to the unedited version except for additional curly textures mimicking Starry Night's swirling patterns.\n\nThese experiments demonstrate that our method can effectively manipulate texture. We will include these results in the updated paper.\n\n### **2. Theoretical Justification for Neural Network Denoisers**\n>The theoretical justification relies on a simplified Patch PCA denoiser, which, while effective, may not fully capture the complex, non-linear dynamics of large-scale neural network denoisers. This simplification could limit the direct applicability of the theoretical claims to state-of-the-art diffusion models.\n\nWe appreciate the opportunity to clarify the relationship between our theoretical analysis and practical neural network denoisers.\n\nOur theoretical framework handles nonlinear settings. Our main result, Theorem A.2, provides closed-form expressions for the optimal denoiser within a broad family of \"admissible local functions,\" which includes both linear and nonlinear functions. This family is motivated by the local score and equivariant score functions introduced in [2] and offers a natural generalization to settings where equivariance is broken—for example, near boundaries. Notably, our analysis reveals that the equivariance condition is redundant; the key structural constraint is what we define as the \"patch shape\" in Definition A.1.\n\nTheorem 5.1 follows as a corollary of Theorem A.2, establishing that when the patch distribution is Gaussian, the optimal denoiser within the admissible local function family coincides with the Patch PCA denoiser. This suggests that even complex neural networks, if locally determined, should approximate this theoretical optimum for Gaussian-like patch distributions during training.\n\nFor neural network denoisers trained on non-Gaussian but approximately Gaussian patch distributions (e.g., close in Wasserstein distance), we hypothesize that the neural network denoisers can be viewed as a bounded perturbation of the optimal Patch PCA denoiser. Based on this assumption, if we denote the neural network denoiser as $f_{NN}$ and Patch PCA denoiser as $f_{PCA}$, assuming $||f_{NN}(x, t) - f_{PCA}(x, t)||_2 \\leq \\epsilon(t)$ and bounds on the Lipschitz constant $L_t$ of flow maps $\\Psi_t$, we can adapt the Alekseev–Gröbner stability argument from [1] to bound trajectory discrepancies. This framework implies that final samples from neural denoisers remain close to those from our theoretical model when the assumptions are satisfied.  \n\n### **3. Editing on More Complex Scenes**\n>While the editing results on CIFAR-10 and ImageNet are promising, the method's effectiveness on more complex scenes with multiple objects or intricate layouts is not fully demonstrated. The compositional control appears to be primarily global, and its precision in more complex scenarios remains an open question.\n\nWe agree that extending the method to multi-object scenes and finer spatial precision is compelling future work. Our primary goal here is to introduce a new Patch PCA filter-based perspective for editing via single manipulation of the initial seed.\n\nFor reference images more complex than the target image, Figure 18 in the Appendix demonstrates editing results for CIFAR-10 using Starry Night as the reference, where our method successfully captures the characteristic bottom-left to middle-right diagonal movement of the reference composition. We will leave a comprehensive evaluation of more complex multi-object scenes as an important direction for future studies.  \n\n---\n\n[1] Benton, Joe, George Deligiannidis, and Arnaud Doucet. \"Error Bounds for Flow Matching Methods.\" TMLR 2024  \n[2] Kamb, and Ganguli. \"An analytic theory of creativity in convolutional diffusion models\". ICML 2025"}, {"review_id": "Jw1xhbQCsC", "reviewer": "Reviewer_1av2", "summary": "The paper presents a methodology to understand the relationship between the initial noise and generated images in diffusion models. Using the principal components of the training dataset patches, the authors find that low-frequency structures in the initial noise are also retained in the generated images. The authors aim to exploit this finding and propose a zero-shot editing algorithm that attempts to modify these structures in the initial noise and control the composition of the final generated image.", "questions": "- Does the proposed Patch PCA translate to diffusion transformers (DiTs)? The convolutional models utilize patch-specific features to perform denoising, but diffusion transformers are not biased towards using local patches. Including DiTs in the analysis could help drive the main argument of the paper further.\n- Why did you choose ImageNet for extracting the principal components, and how would using a different dataset (e.g. FFHQ) affect the analysis and editing?  Since FFHQ is mostly aligned, with faces centered on the images, could you extract different dataset-specific components that the diffusion models learned to compose (e.g. eyes, mouth)?", "limitations": "The only limitation mentioned is that the proposed editing is focused on low-frequency components. However, the most significant limitations would be regarding the applicability of the proposed editing, which has not been discussed thoroughly. The authors should expand their experimental section to demonstrate and address potential limitations of the proposed editing algorithm.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 1, "originality": 2, "strengths_and_weaknesses": "**Strengths**:\n- The main finding of the paper is that the commonly used convolutional diffusion models utilize specific structures in the initial noise when generating images. The authors utilize the principal components of the training dataset patches to probe the initial noise for these patterns and showcase how the diffusion-generated images exhibit similar patterns to the ones highlighted by the PCA components applied on the initial noise. The analysis is straightforward and provides an elegant explanation to an empirical observation that has been previously made regarding the properties of the initial noise and how they are correlated with the generated final image.\n\n**Weaknesses**:\n- The authors aimed to exploit their findings and propose an editing scheme using the extracted PCA components. The PCA-based noise editing requires considerably more experimentation to support and, in the end, hurts the overall presentation of the paper. The authors claim that the proposed zero-shot Patch PCA editing enables compositional control in the generated images. However, the experiments to support this are limited, with the only example being the interpolation between generated images and a 'Target' reference image. The reference image has a very distinct structure, which, although it seems to affect the CIFAR-10 and ImageNet generations, in no way guarantees that the proposed algorithm works in a more general setting. In the supplementary, the authors demonstrate editing with another reference image, but the results in Fig. 18 are nowhere near as successful. To claim compositional control, a larger-scale experiment is necessary, such as the one performed by Li et al. [11], which does numerical composition, i.e., generating K distinct objects in the image.", "was_revised": true, "final_justification": "The paper offers an interesting analysis of how the initial noise in diffusion models affects the contents of the final generated image using the PCA of image patches. The authors also propose an editing algorithm that utilizes the low-frequency structures in the initial noise to guide image generation with a reference image. The image editing algorithm has not been evaluated extensively and would benefit from further experiments on additional models/datasets.", "rebuttal": "We thank the reviewer for the feedback and constructive questions, and for recognizing our analysis that explains how the initial noise correlates with the generated final image.\n\n### **1. On the scale of the editing experiments.**\nThe only weakness raised is the scale of the editing experiments.\n>The authors aimed to exploit their findings and propose an editing scheme using the extracted PCA components. The PCA-based noise editing requires considerably more experimentation to support and, in the end, hurts the overall presentation of the paper. The authors claim that the proposed zero-shot Patch PCA editing enables compositional control in the generated images. However, the experiments to support this are limited, with the only example being the interpolation between generated images and a 'Target' reference image. The reference image has a very distinct structure, which, although it seems to affect the CIFAR-10 and ImageNet generations, in no way guarantees that the proposed algorithm works in a more general setting. In the supplementary, the authors demonstrate editing with another reference image, but the results in Fig. 18 are nowhere near as successful. To claim compositional control, a larger-scale experiment is necessary, such as the one performed by Li et al. [11], which does numerical composition, i.e., generating K distinct objects in the image.\n\nWe think there might be a misunderstanding of our paper's core contribution compared to works like Li et al. [11], which we hope to clarify.\n\nOur paper's primary contribution is a fundamental analysis of the image diffusion model: we identify that low-frequency components of initial noise are key to compositional control. As the reviewer noted, this provides an \"elegant explanation\" of the noise-to-image correlation.\n\nLi et al. [11] leverage the intuition of reliable noise and focus on the downstream application of mining reliable seeds to improve text-to-image generation. While large-scale experiments are crucial for their goal, our editing method is intended as proof-of-concept—a clear demonstration of our core finding about Patch PCA components of initial noise. Its success validates our main thesis about frequency bands.\n\nWe clarify that we define image composition in Lines 32-33 as illumination and major structural lines that contribute to perceived composition, which can be measured by SSIM. The numerical composition is a completely different concept. Whether some principal components from Patch PCA reflect numerical composition is an interesting topic for future study.\n\nIn our response to other reviewers, we have incorporated feedback to:\n1. Add SSIM scores and color histogram correlation to quantitatively evaluate compositional alignment and color preservation (Item 1 in rebuttal to Reviewer t5kH)\n2. Include new experiments on high-frequency noise editing influence (Item 2 in rebuttal to Reviewer t5kH)\n3. Verify findings hold across different deterministic samplers like VE and VP (Item 3 in rebuttal to Reviewer HRUH)\n4. Verify findings hold for flow-matching models (Item 4 in rebuttal to Reviewer HRUH)\n\nGiven that our paper's central scientific contribution is distinct from [11] and the strengthened evaluation, we believe our work makes a valuable contribution to understanding diffusion models.\n\n**Clarification on Compositional Control Success in Figure 18**\n>In the supplementary, the authors demonstrate editing with another reference image, but the results in Fig. 18 are nowhere near as successful\n\nWe respectfully disagree that editing in Fig. 18 is unsuccessful. For Starry Night, the main perceived structure is diagonal movement from bottom left to middle right. Generated images with increasing weights for the reference show a transition from original generation toward variants with diagonal movement similar to the reference, showing compositional structure alignment. We conducted SSIM computation comparing baseline (no edit) and edited case with interpolation factors, showing a monotonic increasing trend:\n\n| Interpolation factors | 0.0 (no edit) | 0.1 | 0.2 | 0.3 | 0.4 | 0.5 | 0.6 | 0.7 | 0.8 | 0.9 | 1.0 |\n|---------------------|---------------|-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|\n| SSIM vs. Reference  | 0.0272        | 0.0741 | 0.1217 | 0.1707 | 0.1915 | 0.2145 | 0.2208 | 0.2309 | 0.2386 | 0.2526 | 0.2614 |\n\nThis quantitatively demonstrates successful progressive alignment toward reference composition.\n\n### **2. On Translating Patch PCA to DiTs**\n>Does the proposed Patch PCA translate to diffusion transformers (DiTs)? The convolutional models utilize patch-specific features to perform denoising, but diffusion transformers are not biased towards using local patches. Including DiTs in the analysis could help drive the main argument of the paper further.\n\nThank you for this insightful question. Yes, our findings and methods apply to transformer-based diffusion architectures like DiTs. Transformer-based diffusion models, such as DiT, have an initial \"Patchify\" step where the image is broken into tokens, creating an inductive bias compatible with our patch-based analysis. Additionally, as shown in recent work [24], different architectures—including U-Nets, U-ViT, and Consistency Trajectory Models—produce remarkably nearly identical images when given the same initial noise (see Fig. 1 and Fig. 11 in [24]). Consequently, both frequency perturbation phenomenon and editing apply to diffusion architecture with transformers that we empirically verify below. \n\nWe couldn't find the DiT pre-trained model on ImageNet-64, FFHQ, or AFHQ, so we used U-ViT instead. We generated 1000 images with identical seeds for EDM and U-ViT, both of which are publicly available pre-trained models on ImageNet and Patch PCA, as well as edited versions using the same edited initial noises, following Figure 1 settings with an interpolation factor of 0.1. We show SSIM and MSE scores between the same seed (paired) and random (unpaired) images.\n\n| Network Pair | SSIM Paired | SSIM Random | MSE Paired | MSE Random |\n|--------------|-------------|-------------|------------|------------|\n| U-ViT vs EDM | 0.8197 ± 0.0794 | 0.0811 ± 0.0522 | 0.0065 ± 0.0045 | 0.1422 ± 0.0657 |\n| U-ViT vs Patch PCA | 0.4805 ± 0.1265 | 0.0266 ± 0.0248 | 0.0542 ± 0.0263 | 0.1248 ± 0.0371 |\n| U-ViT Edited vs EDM Edited | 0.8093 ± 0.0838 | 0.1219 ± 0.0645 | 0.0102 ± 0.0094 | 0.1703 ± 0.0732 |\n\nThese results confirm our Patch PCA based observations and that our editing method applies across different architectures, with edited images showing strong structural alignment regardless of underlying model architecture.\n\n### **3. On the Dataset for Patch PCA Computation**\n\nThe specific dataset used to compute Patch PCA basis is not critical for small patch sizes we use. In Appendix Figure 9, we show that patch covariance matrices computed from ImageNet, FFHQ, and AFHQ are nearly identical for patches smaller than 16×16 (achieving >0.99 correlation). This is because small patches capture universal, low-level statistics of natural images that transcend specific content of any dataset. When using small patches, high-level structures like aligned faces in FFHQ are broken down into generic components (edge detectors, textures), making their statistics highly similar to those from general-purpose datasets like ImageNet. Therefore, we did not observe components specific to features like mouths or eyes.\n\n### **4. On Limitations**\n>The only limitation mentioned is that the proposed editing is focused on low-frequency components. However, the most significant limitations would be regarding the applicability of the proposed editing, which has not been discussed thoroughly. The authors should expand their experimental section to demonstrate and address potential limitations of the proposed editing algorithm.\n\nAs we clarified earlier, our applicability to Fig. 18 is confirmed by SSIM scores. This paper is intended as a proof-of-concept—a clear demonstration of our core finding about Patch PCA components of initial noise. A comprehensive evaluation remains a future goal. \n\n---\n\n[11] Li, Shuangqi, Hieu Le, Jingyi Xu, and Mathieu Salzmann. \"Enhancing compositional text-to-image generation with reliable random seeds.\" ICLR 2024.\n\n[24] Huijie Zhang, Jinfan Zhou, Yifu Lu, Minzhe Guo, Peng Wang, Liyue Shen, and Qing Qu. The emergence of reproducibility and consistency in diffusion models. ICML 2024."}]

Author rebuttal:

Meta-review: This paper investigates the influence of initial noise seeds on the final generation in diffusion models. Through patch-wise PCA analysis, the authors demonstrate that low-frequency components of the initial noise encode universal compositional cues that strongly affect the global structure of generated images.

Reviewers agree that this analysis is **non-trivial and insightful**, and the empirical evidence is clear. The rebuttal provided additional analyses, including extensions to high-frequency components, more quantitative evaluation, and validation across different architectures and samplers, which address most concerns raised by reviewers.

The rating current is two borderline accept, one accept, and one borderline reject.
Reviewer 1av2 maintained a borderline rejection after rebuttal, owing to limited evaluation of the proposed image editing algorithm and requesting broader experiments. However, Reviewer HRUH argued that the editing method is not the main contribution and that the core scientific finding stands on its own. AC agrees with this perspective. Additionally, one review’s final justification is missing, with an earlier rating of borderline accept.

Based on the strength of the main contribution and the quality of the rebuttal, AC recommends acceptance. The authors should incorporate key clarifications and additional results from the discussion into their revision.

-----------------

Lastly, AC would like to suggest that the authors discuss why this phenomenon occurs. Ideally, the initial noise should be pure Gaussian with a signal-to-noise ratio (SNR) close to zero. However, the model is still able to reveal information from the initial noise during inference. Could this originate from the training process? On the one hand, the forward process may not fully perturb the signal into pure noise [a]. On the other hand, since the denoising network is shared across timesteps, later steps expect higher-SNR inputs and may implicitly condition on residual information present in the initial noise. Will this knowledge be shared across timesteps and lead to the current phenomenon?

If it is caused by the SNR problem, will editing the high-frequency information be really possible? If we guarantee the SNR close to zero in training, and let the UNet for the initial denoising timestep not be shared, will the phenomenon still happen?

[a] Common Diffusion Noise Schedules and Sample Steps are Flawed
Decision: Accept (poster)