ScaleDiff: Higher-Resolution Image Synthesis via Efficient and Model-Agnostic Diffusion
Authors: Sungho Koh
SeungJu Cha
Hyunwoo Oh
Kwanyoung Lee
Dong-Jin Kim
Venue: NeurIPS 2025 poster ()
Keywords: Diffusion
High Resolution Image Synthesis
Primary area: applications

Abstract: Text-to-image diffusion models often exhibit degraded performance when generating images beyond their training resolution.
Recent training-free methods can mitigate this limitation, but they often require substantial computation or are incompatible with recent Diffusion Transformer models.
In this paper, we propose ScaleDiff, a model-agnostic and highly efficient framework for extending the resolution of pretrained diffusion models without any additional training.
A core component of our framework is Neighborhood Patch Attention (NPA), an efficient mechanism that reduces computational redundancy in the self-attention layer with non-overlapping patches.
We integrate NPA into an SDEdit pipeline and introduce Latent Frequency Mixing (LFM) to better generate fine details.
Furthermore, we apply Structure Guidance to enhance global structure during the denoising process.
Experimental results demonstrate that ScaleDiff achieves state-of-the-art performance among training-free methods in terms of both image quality and inference speed on both U-Net and Diffusion Transformer architectures.

ScaleDiff: Higher-Resolution Image Synthesis via
Efficient and Model-Agnostic Diffusion
Sungho Koh
SeungJu Cha Hanyang University s ju0220hhnyyang... ac
Hyunwoo Oh
Hanyang University
Hanyang University
ksh0009066haayang ac
komj Sennjiihhanaang ac
Kwanyoung Lee
Dong-Jin Kim
Hanyang University
Hanyang University
mobled37@hanyang ac. kr
dj dkii@hanyyng ac kr
Abstract
Text-to-image diffusion models often exhibit degraded performance when gen- erating images beyond their training resolution. Recent training-fre free methods can mitigate this limitation, but they often require substantial computation or are incompatible with recent Diffusion Transformer models. In this paper, we pro- pose DiffeDiff, a model-agnostic and highly efficient framework for extending the resolution of pretrained diffusion models without any additional training. A core component of our framework is Neighborhood Patch Attention (NPA), an efficient mechanism that reduces computational redundancy in the self-attention layer with non-overlapping patches. We integrate NPA into an SDEdit pipeline and introduce Latent Frequency Mixing (LFM) to better generate fine details. Furthermore, we apply Structure Guidance to enhance global structure during the denoising process. Experimental results demonstrate that ScaleDiff achieves state-of-the of-thh--rtt-ttt- perfor- mance among training- freegg--feeeeee methods in terms of both image quality and inference speed on both U-Net and Diffusion Transformer architectures.
Introduction
Diffusion models have recently emerged as the leading approach in image generation 7 demonstrat- ing the ability synthesize high-fidelity images from simple text prompts 313
21 30) While these models achieve impressive results at standard resolutions (e.g.. under 1024 their performance significantly degrades when generating images at higher resolutions (e.g.. beyond 2048?), often producing artifacts such as repetitive patterns and structural distortions [9] However, training diffusion models directly higher-resolution datasets is prohibitively expensive, requiring both large-scale, high-quality data and substantial computational resources.
As result, recent research has focused extending pre-trained diffusion models to generate higher- resolution images in a training-free free manner [2] g [2
[6]
[7
[9
20
22
23
However, most
of the existing methods are primarily designed for U-Net-based models 1 30 and observe that many existing methods are inapplicable 22 applied to recent Diffusion Transformer (DiT) models [
16 or exhibit limited effectiveness (20 28 when
10 21 29). Figure T highlights this issue, showing clear qualitative differences when existing methods are applied to DiT models. Although patch-based methods g 22 23 such MultiDiffusion (2] inherently architecture-agnosicc and can generate detailed results with DiT models by processing the image in patches at its original trained resolution, they require significant computational redundancy process overlapping patches. This
"Corespooning author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

327
177
430
113
U-Net
1251 sec
843 sec
1465 SEC
407 sec
DiT
Dircet Inference
SDEdit
MultiDiffusion
ScaleDiff (Ours)
Figure 1:
Comparison between U-Net (SDXL) and DiT (FLUX). Zoom in for a better view. Elapsed time generate the image is shown in the top-left corner. Images generated at 40962.
inefficiency creates a major bottleneck for scalable higher-resolution image synthesis in real-world applications, highlighting the need for more efficient and urchitecture-agossie solutions.
In this work, propose DifleDiff, highly efficient and model-agnostic framework for extending the resolution capability of pre-trained diffusion models without any additional training. In particular, we introduce Neighborhood Patch Attention (NPA) to address the computational redundancy inherent in conventional patch- patch-based methods. In self-attention layers, NPA divides the queries into non- overlapping patches and computes attention individually using key and value patches gathered from overlapping spatial neighborhoods. For non-self-attention layers (e.g.. MLP), which are less sensitive resolution, we process the full tensor directly. This design eliminates duplicate computations caused by overlapping image regions, ensuring seamless transitions across patch boundaries. We leverage an iterative bresaple-ddifuse-deee with global semantic coherence. While prior works (20 44 perform upsampling in RGB-space, this often results in oversmoothed outputs and loss of fine details 18 20] To address this, we introduce Latent Frequency Mixing (LFM), which refines the RGB-space upsampled latent by replacing its low-frequency components with those from an alternative upsampling path in latent space. Finally, to further enforce global consistency, we incorporate Structure Guidance (SG) [7 20 39]. Unlike previous approaches that operate RGB CGB-space [20), our method applies SG in the latent space to avoid unnecessary computational overhead. SG reinforces structural coherence by aligning the low-frequency components of the model's intermediate prediction with those from a reference latent.
pipeline [28] to generate higher-resolution images
Our main contributions are summarized as follows: (1) We present ScaleDiff, a model-agnostic frame- work demonstrating state-of-the-an results among training-free free methods for higher-resolution image generation, achieving significant improvements in inference speed on both U-Net and DiT models. (2) We propose NPA, an efficient attention mechanism that reduces computational redundancy by using overlappgg patches in self-attention layers. (3) We introduce LFM, technique integrated with SG to enhance global structural coherence and fine detail synthesis during the denoising process.
2 Related Work
Text-to-Image Generation. Text-to-image generation has advanced rapidly with the development of diffusion models [15
40 41 which generate images by progressively denoising random noise. most notably CLIP 332 -which enable conditioning image generation on natural language prompts. Early methods such as DALL-E 3 and Imagen (36) demonstrated the potential of large-scale language-vision
major driver of this progress has been the integration of powerful text encoders -mort
alignment. The introduction of the Latent Diffusion Model (LDM) 34 further improved efficiency by conducting the diffusion process in lower-dimensional latent space, facilitating practical high-
resolution synthesis. Recent works 5 6 9 10, 21 continue to push the boundaries, exploring

---

alternative generative frameworks like Rectified Flow [26] and architectural innovations such as Diffusion Transformers (DiT) 29 which replace traditional U-Net [35] backbones with transformer-
based architectures 45 to enhance scalability and performance.
Higher Resolution Image Generation. Scaling diffusion models high resolutions often results in repetitive artifacts and structural distortions when naively extrapolated beyond their training resolution [2]. Several methods 66 [1
43 47 address this by fine-tuning or training on high-resolution datasets. Despite these efforts, their scalability remains limited due to the fundamental scarcity of high-resolution data and the sharply increasing training cost with image size.
25
33
To mitigate these challenges, recent work explores training- free strategies 2 8
12
16
17
19 20
22 23 49] that extend the pretrained model'ss resolution methods. 2 g
23 subdivide the target high-resolution images into overlapping trained-resolution patches, which are processed individually
22
and then stitched together. However, it significantly increases computation due the necessary overlap and suffers from object repetition issues. Another line of research [12 T6. 19 49 alters
the internal behavior of the model during inference. For example, ScaleCrafter [22 introduces dilated convolutions into the U-Net to expand its receptive field and reduce repetition. However, these modifications are often architecture-speciff and tend to degrade image quality ultra-high resolutions. Editing-based pipelines [17 [20] generate an image at the model's native resolution, upsample it, and then refine it using techniques such as SDEdit [2 Nevertheless, these editing methods rely on the base model's 's ability to generate strong local details at higher resolutions task that U-Net models can typically handle, but DiT models often struggle with. Compared to prior works, ScaleDiff generates high-resolution images with fine details regardless of the underlying model architecture, while significantly reducing computational overhead.
3 Methods
Given a diffusion model trained on fixed-resolution latents 2 E R our goal is to generate higher-resolution image latents Z Rs where denotes the scaling factor. To achieve this, we propose ScaleDiff, a training-free and model-agnostic framework that efficiently extends pre-trained diffusion models higher resolutions. ScaleDiff consists of two main components. First, we introduce Neighborhood Patch Attention (NPA) (Section (32)), an efficient attention mechanism applicable both U-Net and DiT architectures that enables the processing of higher-resolution latents. Second, we present the ScaleDiff Upscaling Pipeline (Section 3.3 which builds upon the SDEdit framework 2X The pipeline incorporates two key techniques: (i) Latent Frequency
Mixing (LFM), which refines the reference latent to enhance details, and (ii) Structure Guidance
(SG), which maintains global consistency by aligning the low-frequency components of intermediate latent predictions with those of the refined reference.
3.1 Backgrounds
Latent Diffusion Model. Latent diffusion models 229 34 first compress an input image in RGB- space into a lower-dimensional latent 2g E h via an encoder E. This enables subsequent diffusion and denoising to be performed more efficiently in latent space rather than directly on high-resolution pixels. During training, Gaussian noise is gradually added to the clean latent 20 from t = 0 to T. following the forward process:
q(21|20)= = (26 vālzo. (1 - I))
(1)
where { 17 is a set of prescribed noise schedules. A denoising network, often based U- Net (35] or Transformer (45) architecture, is trained to predict the noise added to 2, During inference, sampling starts from random latent 27 (N I). The trained network iteratively predicts the noise and denoises 2, to estimate -,- progressively refining the latent until the final clean representation 20 is obtained. This zg is then decoded the pixel space by a decoder D producing the final image.
Self-attention is key to capturing global context in diffusion models as it allows each token to weigh its interaction with all other tokens. The self-attention output O is formulated as
QKT
0 = softmax
V.
where Q, K, V are the Query, Key, and Value matrices derived by linearly transforming the features extracted from 2, through the network. Especially in transformer architecture, Q, K, V Rxxuxd
3

---

sharing the same spatial dimension as 2, To encode positional information, transformer based diffusion models (29] often incorporate positional encodings 442 into the self-attention mechanism. However, these encodings are tied to the training sequence length. Consequently, when self- attentonnn applied to sequences longer than those seen during training. unseen positional embeddings may disrupt spatial understanding and degrade image quality (27).
Patch-Wiss Denoising. Conventional methods process Z, IBshhaaxxd by directly computing self-attention with Q, K, V E R which results in computational cost of 5' FLOPs. To reduce the significant computational cost and circumvent the resolution limitations introduced by positional encoding. MultiDiffusion (2) divides the input Z into N overlapping patches { where o E Rxx Then, the denoising network is applied individually to each patch 2t to obtain the corresponding denoised patch 2ل_ These individually processed patches are then aggregated to reconstruct the full latent high-resolution Z₁-1 by averaging the values in the overlapping regions.
Specifically, they apply shifted crop sampling strategy with strides S₁ and Sw. corresponding to the height and width dimensions, respectively. As a result, the total number of patches is given by N sh-A
+ +1)
-w--"
+ +1). This leads to Q, K, R--xxxxxd resulting in a total
S4
S
computational cost of Nh FLOPs in self-attention. practice, the stride values are typically set 3) yielding N (2s- 1)2 patches and a corresponding FLOPs of (2s 1) w²d (Table 11.
3.2 Neighborhood Patch Attention
MultiDiffusion circumvents the inherent resolution limitations by decomposing the image into smaller, overlapping patches and processing them individu- ally. They effectively reduce the computational cost in self-attention layers by limiting attention to local regions. However, it often requires substantial over- lap between adjacent patches to ensure smooth tran- sitions at the patch boundaries. This overlap causes non-self-attention layers to require nearly 4x more FLOPs under a common stride setting, compared to single forward pass that processes the full latent once. A detailed breakdown of these computational costs is provided in Table !
Q.K, V getxownd
Paicitty
K,V ua Mstxwudd
To reduce the computational redundancy, we intro- duce Neighborhood Patch Attention (NPA) in Fig- ure 2 Our key insight is that layers such as linear, convolution, and cross-attention perform operations on individual tokens or local regions. Unlike self- attention, these layers remain unaffected by increased input resolution, eliminating the need for patch-based processing. Building on this observation, NPA avoids patch-based processing for these non-self-atteniie layers, allowing them to operate on the full latent tensor Z in a single forward pass. This design elimi- nates redundant computations caused by overlapping patches, thereby keeping the computational cost of non-self-attention layers unchanged (Table ID.
Atlention
0 E R Tn
Figure 2: Process of NPA.
Within the self-attention mechanism, NPA is designed to further mitigate computational overhead by extracting queries from non-overlapping patches. Specifically, given full query tensor Q Igsh swxd we apply shifted crop sampling strategy to obtain set of N query patches {Q.).
where Qi R v xd The crop stride is set to match the patch size, i.e., S₁. 2 S. = resulting
h
in total of N = ch-h/2 + X
sw-ww/2 + 1) = 4s² patches. This ensures that the query patches mn/
do not overlap, keeping the total number of query tokens unchanged. For each non-overlapping query patch Qi, we extract a corresponding key-value patch pair (K Vi) RAxuxd from its spatial neighborhood, using a larger window of size h x centered on Q. Because each K; and V, patch is

---

Table 1: Theoretical FLOPs comparison. NPA reduces the computational complexity of self- attention without affecting the cost non-self-attention operations. k denotes the convolution kernel size, and is the length of text tokens. The Base method represents directly processing the input in single forward pass. MultiDiffusion is calculated based on common stride setting. Symbols highlighted in red indicate key elements for comparison.
Method
Linear
Conv
Cross-Attn
Self-Attn
Base (29
hhwdd (2s-
shold
MultiDiffusion 2
(2x 1)
(2s 1)9toold
(2s
NPA(Ours)
s hwud
8 h
hoold
8
AhowPd
Frequency Domain
Decoded
SDFdit Resalt
Zud
ZR
LIF
Figure 3: Comparison between different reference latents.
drawn from expanded spatial window, the overlap between these patches allows every query patch to attend to a wider context and ensures smooth transitions across patch boundaries. We describe the overall process of the Query, Key, and Value patch extraction algorithm in the supplementary materials.
After that, self-attention is computed between non-overlapping query patch Qi and its corresponding overlapping K/V neighborhood (K, Vi), producing the output O E IR 4 # xd and resulting in h²w² d FLOPs. As this process computed individually on 4s² patches, the final cost is h² w (Table D. Finally, we reassemble the individually computed output patches (O,), into the full attention output tensor O x based on their original spatial positions, which is then passed to subsequent layers (e.g., an MLP block).
3.3 ScaleDiff Upscaling Pipeline
To maintain the global structure of a low-rcsolution image while enhancing high-frequency details during image generation, employ an SDEdit 228]dd pipeline. Starting from a low-resolution image latent 2, we first upscale it to obtain Zny, then inject noise up to the intermediate time step т, and apply denoising using NPA. However, naively applying this process often results in outputs that lack fine texture details and appear overly smoothed [88. This phenomenon arises because upscaling low-resolution image closely resembles the resizing operation used during training. As a result, the model tends to denoise the input toward the distribution of resized training images, rather than synthesizing fine-grained details (00)
To understand this limitation, we compare two upsampling strategies Figure 3 Upsampling directly in latent space yields ZLU, which lacks high-frequency components, resulting in visible artifacts
5

---

ScaleDilT
Lalent Frequency Mixing (LFM)
ZRU
Ziu
Bieubic
LBM
High Freq
Zret
Zeef
Zzy
Zlu
Asi Naric
Bicubic
Low
(p(Donn)))
SCG
Structure Guidance (SG)
Zrat
Ziet
Low Freq
Zeee
Z
X y.)
decoripaic
Zzr
Zaj
Zore
Buler Step
Overview of our pipeline. ScaleDiff starts from generated low-resolution latent, upsamples it with LFM, and diffuses to an intermediate timestep T. each denoising step, network-integen with NPA- applies structure guidance to preserve the global image structure.
Figure 4:
in the decoded image that propagate to the final output. However, because ZLU deviates from the training distribution of RGB-resized images, the subsequent denoising process is not biased toward oversmoothing. In contrast, upsampling RGB space followed by VAE encoding yields ZR which contains rich frequency information and ensures stable, artifact-frre decoding. However, since this process closely mimics training-time resizing operations, it strongly biases the model toward reproducing oversmoothed textures instead of generating fine details.
To leverage the complementary strengths of both approaches, we propose Latent Frequency Mixing (LFM). By combining the low- cow-frequency content from Zzu- steers denoising away from the oversmoothing regime-with the high-frequency content from w ensures stable decoding- we can achieve both sharpness and natural texture. The refined reference latent is:
Zrez LhRo ZLU:
where' ! and à denote low- and nrigh-frequenc components obtained by downsampling and upsampling operations and their residual. This construction guides subsequent denoising toward generating detailed outputs without oversmoothing.
Since NPA processes images through patches, it can introduce repetitive patterns. To mitigate this and enforce global structural consistency, we apply Structure Guidance (SG) following prior work 717 20 33). At each timestep we obtain clean estimate Zoji from the noisy latent Z. and guide it toward Zref by blending their low-frequency components:
Zo1e 2 + (1 -7.) + 7:
where "t controls the guidance strength. This guided prediction Zoje is then utilized to compute the subsequent noisy latent Z,-1. This process steers the generation towards the global structure defined by Zref while allowing the model to synthesize high-frequency details. illutret pipeline.
illustrates

---

SDXL
OSEDiA
DemoFusion usioon
Diffushhigh
FreeScale
ScaleDiit' (Ours)
FLUX
BSRGAN
OSEDiI
ScaleDid (Ours)
"Fresh (a)
(b) "Feast Painting Supper by Giovanat Bartista Teppoio
"ridtng a forest"
apple pie"
Figure 5; Qualitative comparison with other methods. All images are generated at 4096² from the same low-resolution input. Zoom in for better view.
4 Experiments
4.1 Experimental Settings
Implementation Details. We evaluate proposed method, ScaleDiff, both FLUX (2] and SDXL [30] within an iterative 1024" + 2048 + 40962 generation pipeline. For FLUX, we use noise timestep = 600, and structure guidance strength of 7 t. This setup uses 30 denoising steps with guidance scale of 3.5. For SDXL, we set = 400, and the structure guidance strength to Y dy. This configuration uses 50 denoising steps with classifier-fres guidance (CFG) [14 scale of 7.5. All experiments are conducted on a single NVIDIA A6000 GPU.
Baselines. We compare our method with recent training-free methods (ScaleCrafter 12] HiDiffu- sion (49), High 20), FreeScale (31. DemoFusion (8) AceDiffusion v2 (24), resertresol
models (BSRGAN 48 OSEDiff (46)), and training- based model UltraPixel [33). Training-free
7

---

Table 2: Quantitative comparison results. The best results are shown in bold, and the second best results are underlined. All time measurements are expressed in seconds.
Model
Resolution
Method
FTD 4
KID L
15
FID,
KID,
15,
CLIP 1
Time
SDXI. Direct
3D
88.56
0.0124
13.25
58.73
0.0137
20.79
31.57
47
SDXL BSRGAN
48
64.60
0.0041
18.40
41.40
0.0092
23.19
33.03
OSEDif
46
64.79
0.0046
18.89
41.76
0.0094
23.58
32.79
UltraPixel
33
64.61
0.0056
18.58
42,44
0.0093
25.15
32.61
71
ScaleCrafter
12
68.68
0.0033
16.56
43,46
0,0064
23.52
32.07
64
2048
IiDiffussin
49
69.52
0.0040
18.22
42,92
0.0067
24.01
31.50
33
DiffuseHiggh
20
63.27
0.0033
19,10
38.15
0,0062
24.95
32.77
45
SDXL
FreeScale
31
63.50
0.0031
19.06
38.27
0.0062
24.25
32.62
69
AccDiffusion
24
64.86
0.0039
18.37
38.24
0,0068
25.66
32.62
199
Demofasion
8
0.0032
19.15
35,98
0.0050
26.42
32.72
ScaleDiff (Ours)
62.98
0.0032
19,54
38.03
0.0067
25.70
33.11
31
SDXI. Direct
30
182.05
0.0717
7.99
80.80
0.0250
17.68
27.82
328
SDXL BSRGAN
48
64.88
0.0044
18.16
48.97
0,0160
17.04
33.02
SDXL +OEEDaI
46
65.35
0.0045
18.69
45.67
0.01188
17.61
32.88
122
UltraPixel
33
65.39
0.0055
19.08
47.09
0,0112
20.64
32.33
386
ScaleCrafter
12
86.66
0.0110
15.14
79.39
0.0217
14.47
30.25
932
40962
HiDiffusion
49
105.37
0.0216
13.87
112.30
0.0494
12.22
27.21
124
DiffuseHigh
63.91
0.0034
18.99
42,30
0.0079
19.54
32.68
325
FreeScale
311
64.33
0.0036
19.18
39.56
0,0079
18.91
32.56
517
AccDitfusion
64.64
0.0037
18.56
40.92
0.0083
18.42
32.34
1599
Demofusion
g
65.06
0.0041
19.13
41.29
0.0079
19.59
32.61
1005
ScaleDiff (Ours)
0.0025
19.56
0.0080
33.04
FLUXX Direct 21
68.78
0.0069
18.57
42.84
0.0086
22,46
30.79
2048
FLUX BSRGAN
48
64.65
0.0052
19.07
42,01
0.0081
22.98
31.21
FLUX
FLUXX OSEDiff
ScaleDin (Ours)
46
65.10
64.31
0.0056
0.0047
18.51
18.46
41.88
40.03
0.0078
0.0073
23.38
23.25
31.22
31.03
103
FLUX Direct
459.07
0.2775
1.61
367.47 .47
0.2642
1.22
18.03
125]
20963
FLUXX BSRGAN 48
64.76
0.0051
18.84
49.30
0.0125
16.92
31.19
FLUX OSEDiff
64.22
0.0052
19.16
48.37
0.0112
31.13
ScaleDiff (Ours)
64.06
0.0044
18.36
44.29
0.0098
17.41
31.14
407
baselines are evaluated on SDXL, as they are optimized for U-Net architectures. For FLUX, we compare against the base model (natively supports resolutions up 20482) and SR methods.
Evaluation. For quantitative evaluation, we randomly sample 1,000 image-text pairs from the LAION-5B 38 dataset and generate one image per prompt using each method. We compute the Fréchet Inception Distance (FID) 131, Kernel Inception Distance (KID) 44, and Inception Score (IS) 37 between generated images and real images. However, these metrics typically require resizing images to 299² pixels, thereby limiting the evaluation of fine fine-graing grained details. To better assess detail fidelity, we extract multiple patches from each image and calculate patch-level FID,, KID,, and IS, following
g
We also measure the CLIP Score 32 to evaluate text-image alignment.
4.2 Quantitative Comparison
Table compares ScaleDiff with baseline methods for generating images at 2048 and 40962 resolutions. On SDXL, ScaleDiff consistently outperforms existing training- free. training based, and super-resolution methods across key quality metrics, demonstrating its ability to generate highliidelity images. Similar results FLUUX further confirm ScaleDiif''s robustness and model-agnostic design.
ScaleDiff also achieves remarkable inference efficiency. For 40962 resolution on SDXL, requires only 113 seconds-the fastest among training- methods. Compared to the patch-based method Demofusion, ScaleDiff achieves an 8.9x speedup while surpassing it in most evaluation metries, demonstrating NPA's effectiveness. On FLUX, applying NPA yields 3.1x speedup over direct inference at 40962 resolution. While super-resolution models like BSRGAN offer faster inference, they struggle to produce fine details, reflected in lower match-level scores. In contrast, ScaleDiff successfully balances generative quality with computational efficiency.
4.3 Qualitative Comparison
Figure 5 presents qualitative comparison of ScaleDiff with baseline methods for 40962 image generation. While all methods produce high-quality outputs, prior approaches exhibit notable limitations. Super-resolution models like BSRGAN and OSEDiff fail to reproduce fine details, resulting in visibly corrupted facial features (Figure Sp.c). DemoFusion effectively generates fine

---

SDXL
185
239 sec
113
FLUX
890 sec
1148 sec
407 sec
Basc
MultiDiffusion
NPA
Figure Qualitative comparison of replacing NPA in the ScaleDiff pipeline. Inference time for each method is shown in the bottom left. All images are generated at 4096 resolution.
Table 3: Quantitative results of ablation study.
Attention
LFM
FID 4
KID
18
FID,
KID, J
18p
CLIP
Time |
Base
61.91
0.0028
19.47
39.94
0.0082
20.09
33.01
185
MultiDiffusion
61.71
0.0021
19.71
38.08
0.0069
20.85
33.04
239
NPA
61.87
0.0025
19.56
38.89
0,0080
20.41
33.04
113
NPA
64.17
0.0036
19.49
41.55
0.0092
19.41
33.02
113
NPA
62.34
0.0028
19.19
39.49
0.0085
20.16
33.01
113
NPA
64.12
0.0035
18.86
41.50
0.0091
19.71
33.04
113
NPA
61.87
0.0025
19.56
38.89
0.0080
20.41
33.04
113
details but often suffers from repetitive object patterns due to its patch-based processing (Figure DiffuseHigh lacks detailed textures due inherent constraints of RGB-space upsampling (Figure 5a), In contrast, ScaleDiff produces results with improved global structure and finer details, demonstrating superior qualitative performance across different models.
4.4 Ablation Study
Effectiveness of NPA. We validate the effectiveness our proposed Neighborhood Patch Attention (NPA) by comparing it against two alternatives integrated into the ScaleDiff pipeline: (I) direct high-resolution inference (Base) and (2) standard patch-based method (MultiDiffusion) As shown in Figure 6 and Table 3. all methods maintain stable global structure. likely due to the shared ScaleDiff pipeline. However, the Base method produces local artifacts on SDXL and lacks fine details on FLUX, while requiring significantly longer inference times. MultiDiffusion generates high-quality images and achieves best scores, but suffers from substantial computational overhead 148s on FLUUX.. In contrast, our NPA achieves scores comparable to MultiDiffusion while being more efficient (407s on FLUU,, 2.8x speedup), demonstrating an effective balance between generation quality and computational efficiency.

---

(a) LFM X SG x
(b) LFM / SG x
(c) LFM X SG d
(d) LFM SG V
Figure 7: Ablating each component of ScaleDiff. The yellow box highlights the repetition artifacts. All images are generated at 40962 using SDXL. [30]. Zoom for better view.
Table 4: Ablating noise timestep
Model
FID
KID
15
FID,
KID,
1S,
CLIP
700
63.61
0.0036
18.93
37.65
0.0064
25.02
33.01
600
63.60
0.0034
19.20
37.94
0,0067
25.15
33.05
SDXL
500
63.30
0.0033
19.43
38.77
24.42
33,09
400
63.12
0.0032
19.36
38.33
0,0070
24.56
33,07
63.22
0.0032
23.85
33.07
700
64.44
0.0049
18.22
40.37
0.0073
23.62
31.27
FLUX
600
64.45
0.0049
40.76
0.0076
23.16
31.29
500
64.13
0.0049
18.36
41.72
0.0081
22.78
31.29
400
63.59
0.0047
18.45
41.99
0,0083
22.90
31.25
63.67
31.33
Effectiveness of LFM and SG. In Figure and Table 3 we validate the contributions of Latent Frequency Mixing (LFM) and Structure Guidance (SG). When both components are removed (FggZZ,, the model fails to generate coherent results, exhibiting severe object repetition and heavily oversmoothed textures. Adding L.FM alone (Fig. (7) reduces oversmoothing and enables the synthesis of finer details, which is confirmed by improvements in patch-level metries in Table 3 Applying SG alone (Fig- effectively mitigates object repetition, demonstrating its role in enforcing global structural coherence. The full ScaleDiff pipeline (Fig. 70), which combines both LFM and SG, concurrently addresses both issues and achieves the best overall performance among all ablated configurations.
Ablation of Noise Timestep т. The noise timestep т is a critical hyperparameter governing the trade-off between preserving global structure from the upsampled reference image (lower 7) and enabling sufficient synthesis fine- fine-grained details (higher 7). We conduct an ablation study to 400 for SDXL and
determine the optimal for both architectures. As detailed in Table 7 600 for FLUX provide the best balance between structural fidelity and detail generation.
Conclusion
In this paper, we propose ScaleDiff, efficient and model-agnostic framework that enhances the resolution capabilities of pretrained diffusion models without requiring additional training. We introduce Neighborhood Patch Attention (NPA), a mechanism that significantly reduces the computational redundancy typical of traditional patch-bases diffusion approaches. In addition, we propose Latent Frequency Mixing (LFM) and incorporate Structure Guidance (SG) within an gpsample-diffue diffuse-denoise denoise pipeline to improve fine detail synthesis and structural consistency. Our experiments, conducted on both U-Net and Diffusion Transformer architectures, show that ScaleDiff achieves state-of-the-arr performance among training-free methods, delivering superior image quality and faster inference across diverse models. These results highlight ScaleDiff as a powerful and versatile solution for higher-resolution image generation.
10

---

Acknowledgment. This was partly supported by the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korean govemment(MMII) (No.RS- 2020-11201373, Artificial Intelligence Graduate School Program(Hanyang University)) and the Institute of Information &Communications Technology Planning Evaluation (IIP)) grant funded by the Korean government(MSITT Lightweight AI Model for Smart Homes).
Development and Demonstration of
References
[1] S. AL. Stable diffusion: A latent text-to-image diffusion model. https::/sttailiity.oo blog/staae http:/saalleddfffte sifffussoo--ubll-rrlleeaa.. 2022.
[2] O. Bar-Tal L. Yariv, Lipman, and Dekel. Multidiffusion: Fusing diffusion paths for controlled image generation. 2023.
[3] Betker, G. Goh, Jing. T. Brooks, Wang, Li, L. Ouyang, J. Zhuang, J. Lee, Y. Guo, et al. Improving image generation with better captions. Computer Science. https://edn. openai. com/papers/dalll:: pdf. 2(3):8, 2023.
[4] M. Binkowski, J. Sutherland, M. Arbel, and A. Gretton. Demystifying mmd gans. arXiv preprint arXiv; 1801. 01400,, 2018.
[5] J. Chen, J. Yu, C. Ge, L. Yao, E. Xie, Y. Wu, Z. Wang. J. Kwok, P. Luo, H. Lu, et al. Pixart-a: Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint 0rXiv:2310.00422 2023.
[6] J. Chen, C. Ge, E. Xie, Y. Wu, L. Yao, X. Ren, Z. Wang, P. Luo, H. Lu, and Z. Li. Pixart-o: Weak- st-ong training of diffusson transformer for 4k text-to-image generation. In European Conference Computer Vision, pages 74-91. Springer, 2024.
[7] Dhariwal and A. Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-877 8794, 2021.
R. Du, D. Chang, T. Hospedales, Y.-Z. Song, and 7. Ma. Demofusion: Democratising high- resolution image generation with no $$S. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6159-6168, 2024.
[9] P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. Müller, H. Saini, Y. Levi, D. Lorenz, A. Sauer, Boesel, al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024.
[10] P. Gao, L. Zhuo, D. Liu, R. Du X. Luo, L. Qiu, Y. Zhang, C. Lin. R. Huang, S. Geng, et al. Lumina-12x: Transforming text into any modality, resolution, and duration via flow-based large diffusion transformers. arXiv preprint arXiv:2405.055
2024.
[11] I. Guo, Y. He, H. Chen, M. Xia, X. Cun, Y. Wang. S. Huang, Y. Zhang, X. Wang, Q. Chen, et al. Make cheap scaling: A self-cascade diffusion model for higher-resolution adaptation. In European Conference on Computer Vision, pages 39-55. Springer, 2024.
[12] Y. He, S. Yang, H. Chen, X. Cun, M. Xia, Y. Zhang, X. Wang, R. He. Q. Chen, and Y. Shan. Scalecrafter: Tuning-free higher-resolution visual generation with diffusion models. In The Twelfth International Conference on Learning Representations, 2023.
[13] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter. Gans trained by two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
[14] Ho and T. Salimans. Classifier- free diffusion guidance. arXiv preprint 220i 12598. 2022.
[15] Ho, Jain, and Abbeel. Denoising diffusion probabilistic models. Advances neural information processing systems, 2565140-685,,
2020. 2020.

---

[16] L. Huang, R. Fang. A. Zhang, G. Song, S. Liu, Y. Liu, and H. Li. Fouriscale: A frequeney perspective on training-fre free high-resolution image synthesis. In European Conference on Computer Vision, pages 192-212. Springer, 2024.
[17] J. Hwang, Y.-H. Park, and J. Jo. Upsample guidance: Scale up diffusion models without training. arXiv preprint arXiv:2404.01709, 2024.
[18] J. Jeong, S. Han, J. Kim, and S. J. Kim. Latent space super-resolution for higher-resolution image generation with diffusion models. arXiv preprint arXiv:2503. 2025.
[19] Z. Jin, X. Shen, B. Li, and X. Xue. Training-free diffusion model adaptation for variable-sized text-to-image synthesis. Advances in Neural Information Processing Systems, 36.70847-70860, 2023.
[20] Y. Kim, G. Hwang. J. Zhang, and E. Park. Diffusehigh: Training-free progressive high-resolution image synthesis through structure guidance. In Proceedings of the AAAI conference on artificial intelligence, volume 39, pages 4338-4346, 2025.
[21] B. F. Labs. Flux.1: An advanced generative ai model. https://fluxl .io/ 2024.
[22] Y. Lee, K. Kim, H. Kim, and M. Sung. Syncdiffusion:
Coherent montage via synchronized
joint diffusions. Advances in Neural Information Processing Systems, 36:50648 36:50648-50660, 2023.
[23] Z. Lin, M. Lin, M. Zhao, and R. Ji. Accdiffusion: An accurate method for higher-resolution image generation. In European Conference on Computer Vision, pages 38-53. Springer, 2024.
[24] Z. Lin, M. Lin, W. Zhan, and R. Ji. Acediffusion v2: Towards more accurate higher-resolution diffusion extrapolation. IEEE Transactions Pattern Analysis and Machine Intelligence, 2025.
[25] S. Liu, W. Yu, Z. Tan, and X. Wang. Linfusion: 1 gpu, 1 minute, 16k image. arXiv preprint wwXiv2449.02099 2024.
[26] X. Liu, C. Gong, and Q. Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint 2XXiv:200903000 2022.
[27] Z. Lu, Z. Wang, D. Huang, C. Wu, X. Liu, W. Ouyang, and L. Bai. Fit: Flexible vision transformer for diffusion model. arXiv preprint arXiv:2402.12376 2024.
[28] C. Meng, Y. He, Y. Song, J. Song, Wu, J.-Y. Zhu, and S. Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv preprint arXiv:2108.01073. 2021.
[29] W. Peebles and S. Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4195-4201 2023.
[30] D. Podell, Z. English, K. Lacey, A. Blattmann, T. Dockhorn, J. Müller, J. Penna, and R. Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprins arXiv:2307.01952. 2023.
[31] H. Qiu, S. Zhang, Y. Wei, R. Chu, H. Yuan, X. Wang, Y. Zhang, and Z. Liu. Freescale: Unleashing the resolution of diffusion models via tuning- free scale fusion. arXiv preprint 2.Xiv:2412.09626 2024.
[32] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763. PMLR, 2021.
[33] J. Ren, W. Li, H. Chen, R. Pei, B. Shao, Y. Guo, L. Peng, F. Song, and L.. Zhu. Ultrapixel: Advancing ultra-high-resolutie image synthesis to new peaks. arXiv preprint arXiv:2407.02158 2024.
[34] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
12

---

[35] O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. Medical image computing and computer-assister intervention-MICC MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234-241. Springer, 2015.
[36] C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gon- tijo Lopes. B. Karagol Ayan, T. Salimans, al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information processing systems, 35: 36479-36490 2022.
[37] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016.
[38] C. Schuhmann, R. Beaumont, R. Vencu, C. Gordon, R. Wightman, M. Cherti, T. Coombes, A. Katta, C. Mullis, M. Wortsman, al. Laion-5b: An open large-scall dataset for training next generation image-text models. Advances in neural information processing systems, 35: 25278- 22278425294, 2022.
[39] Shi, W. Li, Y. Zhang, He, B. Gong, and Y. Zheng. Resmaster: Mastering high-resolution im- age generation via structural and fine-grained guidance. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 6887-6895, 2025.
[40] J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, pages 2256-2265. pmir, 2015.
[41] J. Song, C. Meng, and S. Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010002502. 2020.
[42] Su, M. Ahmed, Y. Lu, S. Pan, W. Bo, and Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocopputing, 568: 368:127000 170663, 2024.
[43] J. Teng. W. Zheng, M. Ding, W. Hong, J. Wangmi, Z. Yang, and J. Tang. Relay diffusion: Unify- ing diffusion process across resolutions for image synthesis. arXiv preprint arXiv:2309.03350, 2023.
[44] A. Tragakis, M. Aversa,
C. Kaul, R. Murray-Smith, and D. Faccio. Is one gpu enough? pushing image generation at higher-resolutions with foundation models, 2024. URL https: Frrxiv.org/abs/24066 abo/ 07251
[45] A. Vaswani, N. Shazeer, N. Parmar, Uszkoreit, L. Jones, A. N. Gomez, Kaiser, and
I. Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[46] R. Wu, L. Sun, Z. Ma, and L. Zhang. One-step effective diffusion network for real-world image super-resolution. arXiv preprint arXiv:2406.08)) 08177, 2024.
[47] E. Xie, L. Yao, H. Shi, Z. Liu, D. Zhou, Z. Liu, J. Li. and Z. Li. Difffit: Unlocking transferability of large diffusion models via simple parameter-efficient fine-tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4230-4233 2023.
[48] K. Zhang, J. Liang, L. Van Gool, and R. Timofte. Designing practical degradation model for deep blind image super-resolution. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4791 1-900, 2021.
[49] Zhang, Z. Chen, Zhao, Y. Chen, Y. Tang, and J. Liang. Hidiffusion: Unlocking higher- resolution creativity and efficiency in pretrained diffusion models. In European Conference Computer Vision pages 145-161. Springer, 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: All claims made in the paper thoroughly examined, with corresponding explanations provided for each.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
Limitations are provided in supplementary materials
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

Justification:
We detail how FLOPs is calculated in section 3.1 and 3.2
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Detailed explanation of experimental setup is provided, section 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer:
[No]
Justification:
We will release our code after submission.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/t.psscccppbllc/guuiii / Swwelees/sooSSeeeeeeeeeeee
wwtsSubbmiiiioooiiioooooooio for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We do not train any models. Details about evaluation is provided in section 3.1. We cannot share subset for evaluation because we do not hold the rights.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: We build upon prior research in the field, utilizing established evaluation metrics and conducting qualitative analysis for comparison.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
16

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: GPU used for evaluation is provided. Inference time is also provided
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttp::/neurips.. httpr//ps...couuoloiccccccccaaad
Answer: [Yes]
Justification: We utilize a pre-trained diffusion model, which-like any generative model used with caution. In our work, we do not fine-tune or modify the model in any way.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: Our approach preserves the original societal implications of the diffusion model, given that we do not modify its generation behavior.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [No]
Justification: We utilize pre-tratee pre-trained model without making any modifications to its weights.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All sources are properly cited, and the pre-trained model is publicly available for research use.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the batasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [No]
Justification: Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
19

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification:
We used LLM only for editing (e.g., grammar, spelling, word choice).
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy fattps://neur/i: /ttppp////////ooeeeeeee///////// for what should or should not be described.
20

---

A Additional Implementation Details
When generating images with various aspect ratios, we ensure that the longer side of the initial image matches the model's trained resolution. For frequency decomposition, we use spatial downsampling ratio of 4 for FLUX and 8 for SDXL. When evaluating the inference speed of MultiDiffusion (2). the overlap ratio set to 50%, and we use batch size of 16 for SDXL and for FLUX.
In U-Net-based models such as SDXL [30,, the spatial resolution is progressively downsampled across layers. Accordingly, we also downsample the native resolution w)) of NPA match the downsampling ratio at each corresponding layer. In FLUX [21], the MM-DiT architecture [9] is used, where the text tokens concatenated with latent tokens and jointly processed through self-attention layer. Accordingly, in NPA, the text tokens are duplicated for cach patch and concatenated with the corresponding latent tokens within each patch. After the NPA processing, the text tokens are averaged across all patches. In the original setting, text tokens are assigned the position (0, 0) in RoPE 42). When duplicating the text tokens, we assign them the position of the top-left corner of the corresponding Key/Value patch to ensure proper spatial processing.
B Additional Details and Experiments on Neighborhood Patch Attention
B.1 Query Window Random Shifting
While Neighborhood Patch Attention utilizes overlapping key/value patches to ensure a smooth transition at patch boundaries, minor boundary artifacts can sometimes appear the generated output. Query Window Random Shifting is an optional technique designed to further alleviate such artifacts by introducing random variations to the query patch grid at each layer (Figure [8) Specifically, we randomly sample the top and left offsets for padding uniformly from the respective ranges [o, 4
and [0, 21 The query tensor is then zero-padded by total of h in height and w in width using these randomly sampled top-left offsets. Query patches are subsequently extracted from this enlarged canvas. After attention computation, regions corresponding to the added padding are discarded. Since offsets independently resampled at each layer, explicit patch boundaries avoided, which reduces border artifacts with minimal computational overhead. Note that this technique was not used during the evaluation in this paper.
2
Ramdom Padding
Patchify
Q Rshxswxdd
Q (nn=f) (mne
02++0 xaa
Figure Illustration Query Window Random Shifting.
B.2 Comparison of Generation time
Figure prreeents the model processing time various resolutions for SDXL and FLUX, compar- ing three methods: Direct Inference (Base), MultiDiffusion, and NPA. Direct Inference shows quadratic growth processing time as resolution increases, primarily due the cost of global self-attention. MultiDiffusion achieves linear scaling with resolution but suffers from higher baseline overhead, caused by redundant computation on overlapping patches. contrast, NPA eliminates such redundancy and maintains linear scaling, resulting in the lowest processing time.
21

---

Processing Time (N)
Processop 1imn (1)
40
12
Besolution Mullipliar (K)
Resolution Mulliplion
Figure 9: Comparison of model processing time. resolution multiplier of x corresponds to generation at 1024 resolution.
B.3 Panorama Generation with NPA
Our NPA adopts the behavior of MultiDiffusion, making it suitable for wide range of applications. Notably, Figure presenns the results of using NPA for panorama generation FLUX. No other ScaleDiff components were used.
photo of snowy mountain peak with skiers
Natural landscape in anime style illustration
photo of city skyline at night
A photo of desert landscape, golden hour
Figure 10: Panorama Generation with NPA. All images are generated 1024 4096 resolution.
22

---

B.4 Patch Extraction Algorithm of NPA
Algorithm I presents the detailed patch extraction procedure for NPA. Note that query window random shifting (Section B.1) 1) is not included in this algorithm.
Algorithm NPA: Query/Key/Value Patch Extraction
Input: Q E Igmhnwxxd
Full query, key, value tensor b Native height and width Set of non-overlapping query patches Set of overlapping key, value patches
2: Parameters: h. w
3: Output: {Q:)
4:
iK { (V N
5: Nr mh-A//
> Number of patch rows
12/2
6: N: + nw-w/2 +1
> Number of patch columns Total number of patches
w/2
7: N + Nr X No
8: for i 1 N do
9:
hstart + 1i/N,1 x h
> Top-left coordinate of the query patch
2
10:
w start + (i mod N,) x
11:
h2
end f h1 start
Bottom-right coordinate of the query patch
12:
w end t- w stort 2
kkke start t clamp h9
4 0, sh h) 4 O, sw w)
Center K/V patch around query patch
start
we
start clamp 2
> Clamp for window shifting the edge
15:
end
km
f h Start +
wek f- wkk + w
16:
end
17:
Q Q start h2
end w
start w "endd
Non-overlapping query patch extraction Overlapping key, value patch extraction
18:
K, K start h end w start w unnd
19:
V. V start h end start (ndd:))
who who
20:
end for
21:
return (Q.)) (KN (V
C Qualitative Results on Various Models
We present qualitative results of ScaleDiff using SDXL and FLUX across various aspect ratios and resolutions in Figure I1 and Figure 12 To highlight the model-agnostic nature of our method, we also include results using Lumina-T22 10 in Figure 13
D Limitation
ScaleDiff has some limitations. First, tuning-free framework, its performance is inherently constrained by the capabilities of the underlying diffusion model. Second, being a patch-based approach, it relies heavily the diffusion model's 'ss prior knowledge of cropped image regions. This can sometimes lead to inconsistent local content when generating sharp close-up images. Finally, repetitive artifacts may still occur in background regions, common drawback of patch-based generation methods.
23

---

Sataaa Rak Rood Blint With Smokea
New New-dorytic weapy-i--oatiiii Pointing-ldasstee Try
SEURRO
22088 2048
Ancient s worrt aendine forest
cleuring
4096 4096
Vintage British roadster
7R 45
[4096x2048]
[2048x 4096]
dragon flying over mountain londscape
woman aed dress walking field flowers
3072x 4096]
[3022x2048]
Figure 11: Qualitative results of ScaleDiff on SDXL
24

---

Gingerbread house with candles
Annd Woren
[2048X2048]
close-ap porteal HORPAN
standing in
14096 4096
dragon Snaaeiing reauue hoart
(4096X20488
2048X 4096
Brave Senyste hoight for
3072x 4096]
[3072x2048]
Figure 12: Qualitative results of SiffeDiff on FLUX
25

---

photo of golden retriever puppy
Ancient castle rains
[2048x2048]
Close-up portrail of a thongghefal WonGar
4096 4096]
forzest ancien feb dappled sinlight shereol
[4096x2048] Steampunk airshus
[2048x 4096]
close-up of flower
[072x 4096]
[3072x2048]
Figure 13: Qualitative results ScaleDiff on Lumina-T2X
26


Reviews:
[{"review_id": "Jvr7vCjQax", "reviewer": "Reviewer_2Ebs", "summary": "The authors propose a method for extending the resolution of the output images from pretrained diffusion models without additional training. For this, the authors introduce Neighborhood Patch Attention. This helps in reducing computational redundancy in the self-attention layer with non-overlapping patches. They also introduce Latent Frequency Mixing for generating fine-grained details in the output images. Additionally, global consistency is enforced through Structure Guidance, applied in the latent space.", "questions": "1. In the context of upscaling pipeline, how is the latent downsampled and upsampled? Please provide a detailed description of how to compute the high-frequency components.\n2.  The shifted crop sampling strategy should be clearly explained. \n3. The authors need to discuss whether there is a possibility of information loss due to shifted crop sampling. If so, how does the proposed method deal with this?\n3. The authors hypothesize that upscaling the latent in the latent space can mitigate oversmoothing. The basis of this hypothesis needs to be explained. \n4. The process of creating $Z_{latent}$ and the motivation of using it should be explained.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strength\n1. The ideas of avoiding patch-based processing for non-self-attention layers and extracting queries from non-overlapping patches to reduce computation, and latent frequency mixing for image quality enhancement are interesting. \n2. The proposed method can be integrated with pretrained diffusion models without additional training.\n3. The proposed method shows promising results in terms of processing time and image quality.\n\n\nWeakness\n1. In the context of upscaling pipeline, the process of computing the high-frequency components is ambiguous. How is the latent downsampled and upsampled in this context?\n2.  The shifted crop sampling strategy is not clearly explained. The authors have also not discussed whether there is a possibility of information loss due to shifted crop sampling. If so, how does the proposed method deal with this potential problem?\n3. The authors hypothesize that upscaling the latent in the latent space can mitigate oversmoothing. However, the basis of this hypothesis is not described in the paper. \n4. The process of creating $Z_{latent}$ is not clear to me. Does it consist of components from both the RGB space and the latent space? If so, how are these two components blended? What is the motivation behind it?", "was_revised": true, "final_justification": "My concerns are resolved.", "rebuttal": "We are sincerely grateful for your positive and constructive review. Thank you for recognizing the key strengths of our work, particularly the novelty of our Neighborhood Patch Attention (NPA) for efficiency and Latent Frequency Mixing (LFM) for enhancing detail. We have addressed all of your questions below with detailed explanations and new experimental evidence, and we hope these clarifications fully resolve your concerns.\n\n>**Q1: Latent up/downsampling & high-frequency component computation.**  \n\nAs briefly mentioned in Lines 196-198, we use bicubic interpolation for all downsampling and upsampling operations. To compute the frequency components, we follow the approach from prior work \\[1\\]. Specifically, the low-frequency component of a latent (Z) is obtained by downsampling it and then upsampling it back to the original size (i.e., upsample(downsample(Z))). The high-frequency component is then derived by subtracting this low-frequency component from the original latent (Z−Z low ). We acknowledge that this could be stated more clearly and will revise Section 3.3 in the final version to make this process more explicit.\n\n>**Q2: Explanation of shifted crop sampling.**  \n\nAlgorithm 1 in our supplementary material shows the detailed patch extraction procedure for our method. To explain it more intuitively, shifted crop sampling is a patch extraction method analogous to a standard convolution operation; we use a sliding window that moves from the top-left corner of the feature map to crop patches. We acknowledge that this explanation was insufficient in the main text and will revise it in the final version with a direct reference to Algorithm 1\\.\n\n>**Q3: Information loss from shifted crop sampling.**  \n\nWe agree that each patch in patch-by-patch image processing lacks global context and, therefore, can lead to the generation of repetitive objects across the image.  \nIn Figure 1, we have also found these repetitive objects in the existing method (MultiDiffusion).  \nOur ScaleDiff pipeline is specifically designed to address this information loss by using a native resolution reference image for global guidance and using NPA  for efficient local synthesis.  \nFurthermore, as shown in Appendix Table 1,  our proposed NPA outperforms the standard attention mechanism (Base) when integrated into the ScaleDiff Pipeline, confirming the effectiveness of our approach.\n\n>**Q4: Basis for the hypothesis on latent-space upscaling mitigates oversmoothing.**  \n\nTo provide quantitative evidence for our approach, we performed an additional experiment, which will be included in the final paper. The process is as follows:\n\n1. Generate 1K 1024x1024 images to use as a Ground Truth,  \n2. Downsample these images to 512x512, then apply the upsample-diffuse-denoise pipeline to restore them to 1024x1024.  \n3. We compare three methods for upsampling: (1) Latent-space upsampling (Z\\_latent), (2) RGB-space upsampling (Z\\_rgb), and (3) our proposed LFM.  \n4. To evaluate the results, we measure PSNR for fidelity, and Image Entropy and Mean Variance of Laplacian (mVoL) to further evaluate the degree of sharpness. \n\n|  | PSNR↑ | Entropy↑ | mVoL↑ |\n| :---- | :---- | :---- | :---- |\n| Z\\_latent | 20.14 | 7.12 | **525.78** |\n| Z\\_rgb | **23.58** | 7.07 | 49.69 |\n| **LFM (Ours)** | 21.42 | **7.16** | 382.67 |\n\nOur results show that Z\\_latent achieves better Image Entropy and mVoL compared to Z\\_rgb, which demonstrates Z\\_latent mitigates oversmoothing of Z\\_rgb. On the other hand, Z\\_rgb shows better PSNR, representing lower artifacts than Z\\_latent. Our LFM leverages this benefit from both latent features by combining them and shows favorable scores on all the metrics. We will add a table with this analysis to the final version.\n\n>**Q5: The process of creating Z\\_latent and the motivation for using it should be explained.**\n\nFollowing prior work \\[2\\], we create Z\\_latent​  by performing bicubic interpolation on the low-resolution latent Z\\_L​  directly within the latent space.   \nThe motivation for blending Z\\_latent and Z\\_rgb is to combine their complementary characteristics, as analyzed in Appendix C. Our analysis shows that while Z\\_latent avoids an oversmoothing bias, its weak high-frequency component leads to artifacts when decoded. Therefore, LFM leverages the low-frequency components from Z\\_latent as a base to avoid oversmoothing, while replacing its deficient high-frequency components with those from Z\\_rgb to resolve the artifact issue.  \nWe acknowledge that this explanation could be stated more clearly in the main paper. We will revise the manuscript to ensure this mechanism is easier to follow in the final version.\n\n\\[1\\] FreCaS: Efficient Higher-Resolution Image Generation via Frequency-aware Cascaded Sampling, in ICLR 2025  \n\\[2\\] DemoFusion: Democratising High-Resolution Image Generation With No $$$, in CVPR 2024"}, {"review_id": "8LjyPKz680", "reviewer": "Reviewer_e7Kz", "summary": "To address the degradation in image quality when existing open-source text-to-image models generate outputs beyond their training resolution, this paper proposes ScaleDiff, a training-free framework composed of three components: NPA, LFM, and SG. Specifically, NPA eliminates the additional computational overhead introduced by previous patch-based methods, LFM mitigates detail loss caused by upsampling in pixel space, and SG enhances global consistency in the generated images. Qualitative and quantitative experiments validate the effectiveness of the method.", "questions": "I will consider increasing the score once all concerns are addressed.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The paper is well-written with a clear motivation, making it easy to understand.\n2. The proposed NPA is interesting; it is a training-free and model-agnostic attention mechanism that allows effective aggregation of contextual information for each token without introducing additional computational overhead.\n3. The proposed pipeline demonstrates its effectiveness by outperforming several previous approaches when applied to the pretrained SDXL model.\n\nWeaknesses:\n1. The paper claims that previous methods are not applicable to diffusion models based on the DiT architecture, but it does not provide adequate qualitative or quantitative evidence to support this claim. I noticed that Figure 1 displays a qualitative comparison of MultiDiffusion applied to DiT, yet MultiDiffusion is not a recent state-of-the-art method.\n2. While the paper includes comparisons with earlier methods, it would benefit from evaluations against more recent models such as AccDiffusion v2, FreeScale, AP-LDM, and MegaFusion.\n3. Although the proposed method is effective, it lacks in-depth explanations of certain techniques. For instance, why does interpolating high-frequency information in the latent space with LFM help preserve details?\n4. Line 46 of the paper mentions that upsampling in RGB space may lead to over-smoothing, which is indeed a valid concern. However, to the best of my knowledge, most existing methods that perform RGB-space upsampling typically apply a diffusion model for refinement afterward. Therefore, I am concerned that the issue raised by the paper may not be particularly significant.\n5. I noticed that using SG alone may have a negative impact on the model’s output, both in qualitative and quantitative experiments. This raises concerns that the generalizability of SG might be limited.\n6. According to previous studies [1], directly changing the sequence length of the attention inputs in a training-free setting can lead to degraded attention performance. How does NPA address the risk of such degradation?\n\n[1] Training-free Diffusion Model Adaptation for Variable-Sized Text-to-Image Synthesis. NIPS2023.", "was_revised": true, "final_justification": "Most of my concerns have been addressed, except for some clarifications, such as the explanation of why the previous method is difficult to adapt to DiT and the explanation of LFM for helping preserve details. Consequently, I have raised my rating to borderline accept.", "rebuttal": "We are sincerely grateful for your detailed and insightful feedback. We appreciate your recognition of our paper's clear motivation and the novelty of the proposed Neighborhood Patch Attention (NPA). To address your main concerns, we have conducted additional experiments and provided in-depth clarifications for our design choices. We hope these additions and clarifications fully address your concerns.\n\n>**W1: Lack of evidence for inapplicability to DiT**   \n\nThe difficulty in applying existing training-free methods to DiT architectures stems from their foundational designs, which often make them ineffective. Existing training-free methods can be broadly categorized into three groups:\n\n* **Parameter Rectification-based methods** (e.g., ScaleCrafter) typically adjust convolution layers during inference. These methods are fundamentally inapplicable to DiT models without convolution layers.  \n* **SDEdit-based methods** (e.g., Diffusehigh) rely on the base model's ability to generate high-frequency information at high resolutions. As shown in Figure 1, while U-Net models often retain this capability, DiT-based models do not. Accordingly, as can be seen in Figure 1, DiT fails to generate local details when the SDEdit pipeline is applied.  \n* **Patch-based methods** (e.g., DemoFusion) are model-agnostic as they operate by dividing a high-resolution canvas into native-resolution patches for individual processing. This approach, however, introduces significant computational redundancy from handling overlapping regions.\n\nThis leaves patch-based methods as the most applicable option for DiT, yet they suffer from significant computational bottlenecks, a limitation our work is designed to overcome.  \nWe will revise the final version upon acceptance to clearly demonstrate these limitations and add a new figure showing the results of applying prior methods directly to the FLUX (DiT) model.\n\n>**W2: Comparison with more recent SOTA models.**\n\nIn response to your feedback, we have conducted additional experiments comparing ScaleDiff with recent methods, including AccDiffusion v2 (TPAMI 2025\\), FreeScale (ICCV 2025), AP-LDM (arXiv 2024), and MegaFusion (WACV2025) at 2048x2048 resolution.\n\n|  | FID↓ | KID↓ | IS↑ | FID\\_p↓ | KID\\_p↓ | IS\\_p↑ | CLIP↑ | Time(s) |\n| :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- |\n| AccDiffusion v2 | 64.28 | 0.0039 | 18.83 | 39.47 | 0.0075 | 25.82 | 33.03 | 207 |\n| FreeScale | 63.78 | **0.0029** | 18.86 | **37.85** | **0.0060** | 24.65 | 33.08 | 69 |\n| AP-LDM | 63.18 | 0.0033 | 17.40 | 38.99 | 0.0075 | 23.81 | 32.90 | 37 |\n| MegaFusion | 65.42 | 0.0041 | 17.92 | 38.45 | 0.0063 | 23.09 | 33.05 | 30 |\n| ScaleDiff (Ours) | **62.98** | 0.0032 | **19.54** | 38.03 | 0.0067 | **24.70** | **33.11** | 31 |\n\nResults show that our method outperforms recent approaches like AccDiffusion v2, AP-LDM, and MegaFusion.  \nIn addition, our method shows highly competitive results against FreeScale and notably surpasses it in efficiency with a more than 2x faster inference time.  \nDue to the time constraints of the rebuttal period, we will include the full comparison at 4096x4096 resolution as well in the final version to provide a more comprehensive evaluation.\n\n>**W3: In-depth explanation of LFM preserving details.**  \n\nWe provide a detailed analysis and motivation for our Latent Frequency Mixing (LFM) technique in the supplementary material (Appendix C). Our analysis shows that while latent-space upsampling (Z\\_latent) avoids oversmoothing, it lacks the high-frequency components necessary to prevent artifacts when decoded. Conversely, simply using RGB-space upsampled latent leads to an oversmoothing bias in the final result. Therefore, LFM leverages the low-frequency components from Z\\_latent as a base to avoid oversmoothing, while replacing its high-frequency components with those from Z\\_rgb to resolve the artifact issue. We will clarify this explanation in the main paper in the final version upon acceptance to improve readability. \n\n>**W4: Significance of RGB-space upsampling oversmoothing issue.**  \n\nThe core goal of our research is to address the oversmoothing that occurs in the final results **after refining** an RGB-upsampled image with a diffusion model. This is a known challenge that has been previously identified in works like DiffuseHigh \\[1\\] and LSRNA \\[2\\]. Our work tackles this specific problem by introducing Latent Frequency Mixing (LFM), which provides a better-balanced reference latent that is less biased towards a smooth solution. Our ablation study (Table 3, Figure 5b vs 5c) empirically supports this, showing significant improvements in patch-level detail metrics (FIDp​, ISp​) when LFM is applied.\n\n>**W5: Negative impact and limited generalizability of SG.**  \n\nAs we state in our ablation study, applying SG results in a clear mitigation of object repetition. We have demonstrated this in Figure 5(c), which shows a marked reduction in such artifacts compared to the baseline without SG in Figure 5(a). While the quantitative results in Table 3 show a slight drop in one metric (IS) when comparing the baseline (a) to the SG-only version (c), we would like to highlight that SG improves in FID, FID\\_p, IS\\_p, and the CLIP score, demonstrating its positive effect on image fidelity and text alignment. ​Most importantly, our results in Table 3 show that the complete ScaleDiff pipeline (LFM+SG) achieves the best overall performance, outperforming the LFM-only version (b). This demonstrates that SG is an indispensable component for improving global structure.\n\n\n>**W6: How NPA avoids attention degradation from sequence length changes.**  \n\nOur proposed method does not suffer from the performance degradation issue mentioned in \\[3\\]. The problem described in \\[3\\] arises from an increase in the number of key/value tokens during the attention calculation at higher resolutions. However, our NPA is designed to maintain the number of key and value tokens for each query to be identical to that of the native resolution. Since the number of tokens per attention operation does not change, the performance degradation is effectively avoided.\n\n\\[1\\] DiffuseHigh: Training-free Progressive High-Resolution Image Synthesis through Structure Guidance, in AAAI 2025  \n\\[2\\] Latent Space Super-Resolution for Higher-Resolution Image Generation with Diffusion Models, in CVPR 2025  \n\\[3\\] Training-free Diffusion Model Adaptation for Variable-Sized Text-to-Image Synthesis, in NeurIPS 2023\\."}, {"review_id": "hieUJBfoMm", "reviewer": "Reviewer_YkCW", "summary": "This paper presents ScaleDiff, a training-free, resolution-agnostic framework for extending the generation resolution of pretrained text-to-image diffusion models, including both U-Net and Diffusion Transformer architectures. The method introduces Neighborhood Patch Attention (NPA) to reduce redundancy in self-attention over non-overlapping patches, significantly improving efficiency over standard patch-based approaches. Additionally, Latent Frequency Mixing (LFM) enhances fine detail synthesis, while Structure Guidance improves global coherence during denoising. Integrated into an upsample-diffuse-denoise pipeline, ScaleDiff achieves state-of-the-art performance in image quality and inference speed compared to other training-free high-resolution generation methods.", "questions": "+ Include some analysis or visual examples of where ScaleDiff fails or degrades (e.g., extreme compositions, long prompts, or cluttered scenes). How does the system behave under such inputs?", "limitations": "No other limitations", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strength**\n\n+ Works with both U-Net and Diffusion Transformer backbones, broadening applicability beyond methods tailored only for U-Nets.\n\n+ The proposed Neighborhood Patch Attention (NPA) effectively reduces redundant computations while preserving receptive field coverage, improving speed and scalability.\n\n+ Significant improvements in FID and CLIP scores on multiple datasets and backbones. Visuals show noticeably better high-frequency details.\n\n**Weaknesses**\n\n+ The porposed techniques (NPA, LFM, SG), while efficient in theory, may require non-trivial engineering to integrate into existing pipelines as they looks quite complex to implement, especially NPA.\n\n+ Lack of discussion about failure cases.", "was_revised": true, "final_justification": "The author address my concerns well but I still believe this work is not strong enough to guarantee a score of **5** so I keep original score of **4**.", "rebuttal": "We are sincerely grateful for your valuable feedback and positive evaluation. We are encouraged that you recognized our method's broad applicability across different architectures and the efficiency of our Neighborhood Patch Attention (NPA). We have carefully addressed the concerns you raised in our detailed response below and hope it fully clarifies the points you raised.\n\n>**W1: Implementation complexity of proposed techniques.**  \n\nWe understand this concern. To facilitate adoption and verify our implementation's simplicity, we will release our code as open-source as well as the supplementary material upon publication. We are confident that this will demonstrate the ease of integrating ScaleDiff into existing diffusion model frameworks.\n\n>**W2/Q1: Discussion of failure cases.**  \n\nThank you for the suggestion. We have included a \"Limitations\" section in Appendix E, which discusses some general failure cases, such as inconsistent local content in close-ups and repetitive background textures. In response to your suggestion, we have also experimented with the examples you mentioned (extreme compositions, long prompts, cluttered scenes). Our analysis shows that since the global structure of the final image is guided by the initial image generated at the native resolution, the performance in these challenging scenarios is largely dependent on the capabilities of the base model itself. We will add a new figure to the final version illustrating these results to provide a more comprehensive understanding of our method's limitations."}, {"review_id": "derNAVwPYP", "reviewer": "Reviewer_YvLk", "summary": "This paper focuses on extending existing diffusion models for higher-resolution image generation in a training-free manner. Specifically, the authors propose ScaleDiff, a novel approach capable of scaling up the generation resolution of diffusion models based on both U-Net and DiT. The framework incorporates two key components: Neighborhood Patch Attention (NPA) and Latent Frequency Mixing (LFM). Experimental results demonstrate that the proposed ScaleDiff efficiently achieves higher-resolution image generation while exhibiting better performance.", "questions": "Please refer to the 'Weaknesses' section for details.\n\nMy primary unresolved concerns focus on the following aspects:\n1. **More comprehensive and convincing evaluations**.\n2. **Further polish on figures and presentations**.\n3. **Some essential baseline methods**.\n4. **More related literature expected to be discussed and compared**.\n\nSince the overall quality of the paper is relatively impressive, I think I will be glad to raise my rating if the authors can address my concerns.", "limitations": "Yes, the authors make an attempt to solve the existing limitations.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**:\n\n1. **Writing and Presentation**: The paper is clear, well-written, and easy to follow.\n2. **Concise and Effective Design**: The proposed ScaleDiff is a training-free approach that can be broadly applied to existing diffusion models based on U-Net and DiT.\n3. **Outstanding Performance**: Experimental results demonstrate that ScaleDiff can efficiently generate higher-resolution images of high quality, illustrating its effectiveness and efficiency.\n4. **Necessary Ablation Studies**: The paper includes essential ablation studies to validate the effectiveness of the proposed key modules, making the results more convincing.\n5. **Comprehensive Visualization**: The authors also provide additional qualitative results at various resolution scales in the supplementary material, further confirming the method's strong scalability and promising application potential.\n\n**Weaknesses**:\n\n1. **Limited Evaluation Scope**: The current quantitative evaluation is conducted on 1,000 samples from LAION, which is somewhat limited and may introduce performance variance. Thus, a larger-scale evaluation testbed is expected to make the results more robust and convincing.\n2. **More Comprehensive Evaluation and Metrics**: While the authors provide synthesis results at other scales, quantitative results (similar to ScaleCrafter) at extreme aspect ratios (e.g., 4:1 and 1:4) could further highlight ScaleDiff’s universality. Moreover, beyond FID and patch-based FID, metrics like KID and patch-based KID are also widely adopted for evaluating higher-resolution image generation performance.\n3. **Figures and Explanation**: I recommend refining Figure 3 to reduce blank space and improve layout efficiency. Additionally, it would be helpful to explicitly illustrate how low-frequency and high-frequency components are derived in the diagram (readers might intuitively assume bandpass filtering, while the method actually uses downsampling+upsampling). Furthermore, the hypothesis in Line 189 is confusing: why do imperfect images generated via latent-space upsampling provide high-quality high-frequency components? A deeper explanation, either theoretical or experimental, would help readers better understand this subsection.\n4. **More Key Baselines**: The paper would benefit from including comparisons with more baseline approaches. For instance, replacing BSRGAN with StableSR [1*] or ESRGAN [2*], as well as conventional approaches like Bilinear/Bicubic interpolation, is expected to better demonstrate the advantages of the proposed ScalDiff.\n5. **More Related Work to be discussed**: The paper would benefit from a more thorough discussion and comparison with recent advances in high-resolution image/video generation, such as URAE [3*], MegaFusion [4*], BeyondScene [5*], ResAdapter [6*], FreeScale [7*], and LinFusion [8*]. (especially those post-processing and tuning-free ones)\n\n[1*] StableSR: Exploiting Diffusion Prior for Real-World Image Super-Resolution, in IJCV 2024.\n\n[2*] ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks, in ECCV workshop 2018.\n\n[3*] Ultra-Resolution Adaptation with Ease, in ICML 2025.\n\n[4*] MegaFusion: Extend Diffusion Models towards Higher-resolution Image Generation without Further Tuning, in WACV 2025.\n\n[5*] BeyondScene: Higher-Resolution Human-Centric Scene Generation With Pretrained Diffusion, in ECCV 2024.\n\n[6*] ResAdapter: Domain Consistent Resolution Adapter for Diffusion Models, in AAAI 2025.\n\n[7*] FreeScale: Unleashing the Resolution of Diffusion Models via Tuning-Free Scale Fusion, in arXiv 2024.\n\n[8*] LinFusion: 1 GPU, 1 Minute, 16K Image, in arXiv 2024.", "was_revised": true, "final_justification": "I have carefully read the other reviewers' comments and the authors' responses. \nI believe the authors have addressed most of my concerns, and based on my pre-rebuttal assessment of this paper, I consider it to be a work of good overall quality, in terms of novelty, core contributions, and presentations. \nTherefore, I have adjusted my final score accordingly. By the way, I hope the authors can further improve their paper based on all reviewers' suggestions, including discussions and comparisons of related work, as well as clearer presentation. \nAdditionally, I hope, in the revised version, the authors can complete those experiments that couldn't be finished within the limited rebuttal period, as they could further enhance the paper's overall quality and contribution.", "rebuttal": "Thank you for your comprehensive and constructive feedback. We are grateful for your positive assessment, particularly your recognition of our method's effective design and outstanding performance. In response to your valuable suggestions, we have conducted additional experiments to provide a more comprehensive evaluation against new metrics and baselines. We hope that these results and our detailed responses fully address your concerns.\n\n>**W1: Limited Evaluation Scope**  \n\nWe acknowledge that evaluating with 1,000 samples has its limitations. However, for fair comparison with \\[1,2,3\\],  we use 1,000 samples to follow the common practice established by the prior works.\n\n>**W2: More Comprehensive Evaluation and Metrics**  \n\nThank you for the constructive feedback. In response to your suggestion, we have conducted additional experiments to include KID and patch-based KID metrics (KID\\_p). The results are as follows:\n\n| Model | Resolution | Method | KID↓ | KID\\_p↓ |\n| ----- | ----- | :---- | :---- | :---- |\n| SDXL | 2048 | SDXL Direct | 0.0124 | 0.0137 |\n|  |  | SDXL \\+ BSRGAN | 0.0041 | 0.0092 |\n|  |  | UltraPixel | 0.0056 | 0.0093 |\n|  |  | ScaleCrafter | 0.0033 | 0.0064 |\n|  |  | HiDiffusion | 0.0040 | 0.0067 |\n|  |  | DiffuseHigh | 0.0033 | 0.0062 |\n|  |  | DemoFusion | 0.0032 | **0.0050** |\n|  |  | ScaleDiff (Ours) | **0.0032** | 0.0067 |\n|  |  |  |  |  |\n|  | 4096 | SDXL Direct | 0.0717 | 0.0250 |\n|  |  | SDXL \\+ BSRGAN | 0.0044 | 0.0160 |\n|  |  | UltraPixel | 0.0055 | 0.0112 |\n|  |  | ScaleCrafter | 0.0110 | 0.0217 |\n|  |  | HiDiffusion | 0.0216 | 0.0494 |\n|  |  | DiffuseHigh | 0.0034 | 0.0079 |\n|  |  | DemoFusion | 0.0041 | **0.0079** |\n|  |  | ScaleDiff (Ours) | **0.0025** | 0.0080 |\n|  |  |  |  |  |\n| FLUX | 2048 | FLUX Direct | 0.0069 | 0.0086 |\n|  |  | FLUX \\+ BSRGAN | 0.0052 | 0.0081 |\n|  |  | ScaleDiff (Ours) | **0.0047** | **0.0073** |\n|  |  |  |  |  |\n|  | 4096 | FLUX Direct | 0.2775 | 0.2642 |\n|  |  | FLUX \\+ BSRGAN | 0.0051 | 0.0125 |\n|  |  | ScaleDiff (Ours) | **0.0044** | **0.0098** |\n\nOur ScaleDiff demonstrated favorable performance on the newly added KID and KID\\_p metrics as well.  \nDue to the limited time for the rebuttal, we will include the evaluation on extreme aspect ratios in the final version of the paper.\n\n>**W3: Figures and Explanation**  \n\nThank you for these concrete suggestions. We will revise Figure 3 to be more space-efficient and to explicitly illustrate that the frequency components are derived via downsampling-upsampling, not bandpass filtering.\n\nRegarding the LFM hypothesis,  we provide both a detailed analysis in Appendix C and new quantitative evidence from an experiment designed to justify the hypothesis. In the new experiment, we compare three upsampling methods: (1) Latent-space upsampling (Z\\_latent), (2) RGB-space upsampling (Z\\_rgb), and (3) our proposed LFM. The detailed experimental setting can be found in our response to Reviewer 2Ebs (Q4). We measure PSNR for fidelity and Image Entropy and Mean Variance of Laplacian (mVoL) to evaluate sharpness and the degree of oversmoothing.  \nThe results are as follows:\n\n|  | PSNR↑ | Entropy↑ | mVoL↑ |\n| :---- | :---- | :---- | :---- |\n| Z\\_latent | 20.14 | 7.12 | **525.78** |\n| Z\\_rgb | **23.58** | 7.07 | 49.69 |\n| **LFM (Ours)** | 21.42 | **7.16** | 382.67 |\n\nZ\\_rgb achieves a higher PSNR, but its extremely low Entropy and mVoL scores quantitatively demonstrate the oversmoothing problem. In contrast,  Z\\_latent, despite having a lower PSNR due to artifacts, achieves significantly higher Entropy and mVoL scores, confirming it effectively mitigates oversmoothing.  \nLFM leverages  Z\\_latent as a base to avoid oversmoothing, while replacing its high-frequency components with those from Z\\_rgb to resolve the artifact issue.  \nWe will add this table and a clearer explanation to the final manuscript.\n\n>**W4: More Key Baselines**  \n\nThank you for the suggestion. We have conducted additional comparisons with ESRGAN and Bicubic interpolation:\n\n| Model | Resolution | Method | FID↓ | KID↓ | IS↑ | FID\\_p↓ | KID\\_p↓ | IS\\_p↑ | CLIP↑ |\n| ----- | ----- | :---- | :---- | :---- | :---- | :---- | :---- | :---- | :---- |\n| SDXL | 2048 | ESRGAN | 65.38 | 0.0048 | 17.83 | 41.97 | 0.0093 | 22.01 | 33.01 |\n|  |  | Bicubic | 64.20 | 0.0039 | 18.84 | 49.08 | 0.0162 | 21.52 | 33.05 |\n|  |  | ScaleDiff (Ours) | **62.98** | **0.0032** | **19.54** | **38.03** | **0.0067** | **25.70** | **33.11** |\n|  |  |  |  |  |  |  |  |  |  |\n|  | 4096 | ESRGAN | 65.43 | 0.0048 | 17.65 | 49.31 | 0.0158 | 16.63 | 33.01 |\n|  |  | Bicubic | 64.20 | 0.0039 | 18.88 | 50.31 | 0.0161 | 18.23 | **33.05** |\n|  |  | ScaleDiff (Ours) | **61.87** | **0.0025** | **19.56** | **38.89** | **0.0080** | **20.41** | 33.04 |\n|  |  |  |  |  |  |  |  |  |  |\n| FLUX | 2048 | ESRGAN | 65.01 | 0.0053 | 18.87 | 41.32 | 0.0078 | 23.14 | 31.16 |\n|  |  | Bicubic | 64.56 | 0.0054 | **19.41** | 42.69 | 0.0101 | 22.64 | **31.26** |\n|  |  | ScaleDiff (Ours) | **64.31** | **0.0047** | 18.51 | **40.03** | **0.0073** | **23.38** | 31.22 |\n|  |  |  |  |  |  |  |  |  |  |\n|  | 4096 | ESRGAN | 65.01 | 0.0053 | 18.87 | 48.88 | 0.0119 | 17.29 | 31.15 |\n|  |  | Bicubic | 64.56 | 0.0054 | **19.30** | 54.24 | 0.0190 | 16.84 | **31.26** |\n|  |  | ScaleDiff (Ours) | **64.06** | **0.0044** | 18.36 | **44.29** | **0.0098** | **17.41** | 31.14 |\n\nAs shown in the table, our method provides a clear advantage over these baselines as well. Traditional upscalers like ESRGAN and Bicubic often fall short in generating the fine details. This is strongly reflected in the patch-level fidelity metrics.\n\n>**W5: More Related Work to be discussed**  \n\nThank you for providing a list of relevant recent works. We will expand our related work section in the final manuscript to thoroughly discuss these advances.   \nFor instance, FreeScale enhances overall structure and detail via Scale Fusion, which mixes frequencies from global and local attention. However, its local attention calculation introduces computational redundancy, an issue our NPA is specifically designed to solve. Furthermore, FreeScale's reliance on dilated convolution makes it incompatible with DiT architectures.  \nMegaFusion employs a truncate and relay strategy that continues the denoising process at a higher resolution from an intermediate low-resolution step. This differs from our upsample-diffuse-denoise pipeline, requiring fewer total denoising steps. However, MegaFusion is not demonstrated at very high resolutions like 4096x4096  \nFor a quantitative comparison with some of these recent methods, please refer to our response to Reviewer e7Kz (W2).\n\n\\[1\\] DemoFusion: Democratising High-Resolution Image Generation With No $$$, in CVPR 2024  \n\\[2\\] Is One GPU Enough? Pushing Image Generation at Higher-Resolutions with Foundation Models, in NeurIPS 2024  \n\\[3\\] DiffuseHigh: Training-free Progressive High-Resolution Image Synthesis through Structure Guidance, in AAAI 2025"}, {"review_id": "adrNRVDWW3", "reviewer": "Reviewer_BUQE", "summary": "This paper introduces **ScaleDiff**, a training-free, model-agnostic framework for extending the resolution of pretrained diffusion models beyond their original training size. \n\nThe key component, **Neighborhood Patch Attention (NPA)**, efficiently reduces redundancy in self-attention by using non-overlapping patches, making it suitable for both U-Net and Diffusion Transformer architectures.\n\n Combined with **Latent Frequency Mixing (LFM)** for fine-detail generation and **Structure Guidance** for improving global structure, ScaleDiff achieves state-of-the-art image quality and inference speed among training-free upsampling methods.", "questions": "### Questions and Suggestions for the Authors\n\n**Q1: Clarify and support the claims on inference speed and performance.**  \n- Table 2 shows only marginal performance gains or even worse results compared to FLUX+GAN, and the inference speed is notably slower.  \n- Could the authors provide a more detailed and transparent comparison, including runtime benchmarks under consistent conditions?  \n- Please clarify or temper the claims regarding state-of-the-art speed and performance if the evidence does not fully support them.\n\n**Q2: Explain the notation and role of `Z_rgb` clearly.**  \n- The notation `Z_rgb` is ambiguous—is it a latent representation or raw RGB pixel data?  \n- Given that other `Z_` variables denote latent features, please clarify this inconsistency to avoid reader confusion.  \n- Consider adding a dedicated explanation or diagram showing how `Z_rgb` differs from or relates to `Z_latent`.\n\n**Q3: Justify the construction of the reference latent `Z_ref` as the sum of `Z_high` and `Z_low`.**  \n- What is the motivation behind this design choice?  \n- Would using the actual RGB image as the reference latent provide a clearer or stronger supervisory signal?  \n- Please discuss alternatives and provide ablation studies or analyses demonstrating why this approach is preferable.\n\n**Q4: Explain the large difference in timestep parameter `tau` between backbones.**  \n- Why is `tau=0.6` used for FLUX while `tau=400` is used for SDXL?  \n- Since timesteps are typically integers, please clarify what these values represent and why such a large gap is necessary.  \n- Providing more details on how `tau` is chosen and tuned for different backbones would improve reproducibility.\n\n**Q5: Improve qualitative results and visual assessments.**  \n- The generated images in Figure 4 appear random, making it difficult to visually assess improvements.  \n- Could the authors provide additional visual comparisons with controlled conditions or include quantitative perceptual metrics to better demonstrate quality differences?", "limitations": "No, they didn't discuss.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths**  \n- The paper is complete and generally clear for a broad readership.  \n- The method does not require costly retraining, making it more practical.  \n- Experimental results indicate that the approach is promising.\n\n**Weaknesses**  \n- The claim of state-of-the-art inference speed and performance is not fully supported. In Table 2, performance improvements over competing methods are marginal or absent, and in some cases, ScaleDiff performs worse than FLUX+GAN. Additionally, the reported inference speed is significantly slower than FLUX+GAN, which contradicts the stated contributions.  \n- The notation for `Z_rgb` is confusing. It is unclear whether this denotes a latent representation or pixel-level RGB data. Since the main text consistently treats `Z_` variables as latents, this inconsistency reduces clarity.  \n- The motivation behind constructing `Z_ref` by adding `Z_high` and `Z_low` is unclear. Wouldn’t using the actual RGB image as a reference latent provide a clearer supervisory signal than this manipulated combination?  \n- There is a large discrepancy in the `tau` parameter between the FLUX backbone (`tau=0.6`) and SDXL (`tau=400`). Since timestep values are typically integers, the reason for such a wide gap should be explained.  \n- The generated images in Figure 4 appear random, making it difficult to visually assess the superiority of the proposed method.  \n- Minor issue: there is a typo “SclaeDiff” in line 92.", "was_revised": true, "final_justification": "The rebuttal addressed some of my concerns, but not fully satisfactorily. While Lines 53–55 mention the focus on \"training-free\" methods, Table 2 does not clearly indicate which of the compared methods fall under this category. It remains unclear whether some of the bolded values correspond to training-free approaches or pretrained models, for instance, are methods like DemoFusion and UltraPixel training-free?\n\nRegarding qualitative results, I expected to see more controlled comparisons, for example, using the same random seed and input conditions, to better evaluate whether the proposed method provides a visual improvement that aligns with the quantitative metrics. This is especially important when comparing with methods that perform upscaling from low-resolution inputs.\n\nOverall, based on these unresolved issues and after reading the other reviews, I am inclined to maintain my original score.", "rebuttal": "Thank you for your detailed feedback and for raising several important questions. We appreciate that you recognized our method's practicality in avoiding costly retraining. We have carefully addressed your concerns below, focusing on clarifying our core claims and experimental methodology. We hope these explanations provide the necessary clarity and fully resolve your concerns.\n\n>**Q1: Clarify and support the claims on inference speed and performance.**  \n\n Please note that BSRGAN is a “trained” super-resolution model. As stated in our introduction (Lines 53-55), we would like to clarify that our model is the SOTA and the fastest among the “training-free” high-resolution generation methods. Also, while BSRGAN is faster, our model significantly outperformed BSRGAN both qualitatively (Figure 4a) and quantitatively (44.29 vs. 49.30 in patch-level FID score in Table 2). \n\n>**Q2: Explain the notation and role of Z\\_rgb clearly.**  \n\nIn this paper, we denote Z (both Z\\_rgb and Z\\_latent) as a latent representation. As illustrated in Figure 3, Z\\_rgb is a latent representation obtained by first decoding a low-resolution latent into the RGB space, upsampling it in the RGB space, and then re-encoding it back into the latent space. Z\\_latent is the result of directly upsampling the low-resolution latent in the latent space. We will revise section 3.3  in the final version upon acceptance to ensure clarity and prevent any misunderstanding.\n\n>**Q3: Justify the construction of the reference latent Z\\_ref as the sum of Z\\_high and Z\\_low.**  \n\nThe motivation and a detailed analysis of our Latent Frequency Mixing (LFM) technique are provided in the supplementary material (Appendix C). Our analysis shows that using Z\\_rgb as the starting point for the diffuse-denoise process leads to an oversmoothing bias in the final image. Conversely, Z\\_latent successfully avoids the oversmoothing bias. However, it lacks sufficient high-frequency components, which causes noticeable artifacts whend decoded.  \nLFM leverages the low-frequency components from Z\\_latent as a base to avoid oversmoothing, while simultaneously replacing its deficient high-frequency components with those from Z\\_rgb to resolve the artifact issue.\n\nOur paper's ablation study shows that applying LFM mitigates oversmoothing compared to using RGB image as the reference (Z\\_rgb). This is demonstrated visually in Figure 5 (a) vs. (b) and supported quantitatively by significantly better scores on detail metrics in Table 3 (a) vs. (b).\n\nTo provide further quantitative evidence of LFM, we conducted an additional experiment comparing the reference latents. We measure PSNR for fidelity and Image Entropy and Mean Variance of Laplacian (mVoL) to evaluate sharpness and the degree of oversmoothing. For the full experimental setup, please refer to our response to Reviewer 2Ebs (Q4).\n\n|  | PSNR↑ | Entropy↑ | mVoL↑ |\n| :---- | :---- | :---- | :---- |\n| Z\\_latent | 20.14 | 7.12 | **525.78** |\n| Z\\_rgb | **23.58** | 7.07 | 49.69 |\n| **LFM (Ours)** | 21.42 | **7.16** | 382.67 |\n\nThe table quantitatively confirms the oversmoothing issue with the Z\\_rgb reference (indicated by its low Entropy and mVoL scores) and Z\\_latent avoids this issue. Our LFM successfully combines the strengths of both, achieving favorable scores across all metrics.\n\n>**Q4: Explain the large difference in timestep parameter tau between backbones.**  \n\n SDXL is a traditional diffusion model that uses discrete integer timesteps (typically from 1 to 1000), while FLUX  is a Rectified Flow model that operates on a continuous normalized timestep from 0 to 1\\. Our notation for FLUX (τ=0.6) represents the starting point of the denoising process as 60% along the continuous-time trajectory. For SDXL, τ=400 means starting from the 400th discrete step. We will clarify the notation in the final version upon acceptance.  \nThe choice of timestep τ is a critical hyperparameter that balances two competing factors: preserving the global structure of the upsampled image versus allowing the model sufficient freedom to synthesize fine-grained details. The values we selected were determined empirically to find an optimal trade-off point. To thoroughly validate this choice and enhance reproducibility, we will provide a full ablation study in the final version—including both qualitative and quantitative results—to demonstrate the impact of varying τ on generation quality.\n\n>**Q5: Improve qualitative results and visual assessments.**  \n\nWhen comparing against methods that generate images directly at high resolution (e.g., ScaleCrafter), using the same seed does not guarantee similar outputs. This technical challenge is also reflected in the figures of prior works \\[1, 2, 3\\]. However, for methods that are based on upscaling a low-resolution image (e.g., DiffuseHigh, DemoFusion, and ours), a direct comparison from the same starting image is possible. We will add a new figure with such a controlled comparison in the final version upon acceptance to better highlight the qualitative advantages of our method.\n\n>**Minor Issue: Typo \"SclaeDiff\" in line 92\\.**  \n\nThank you for catching this. We will correct it and also carefully check for any other typos throughout the paper.\n\n\\[1\\] DemoFusion: Democratising High-Resolution Image Generation With No $$$, in CVPR 2024  \n\\[2\\] Is One GPU Enough? Pushing Image Generation at Higher-Resolutions with Foundation Models, in NeurIPS 2024  \n\\[3\\] FreCaS: Efficient Higher-Resolution Image Generation via Frequency-aware Cascaded Sampling, in ICLR 2025"}]

Author rebuttal:

Meta-review: ScaleDiff proposes a training-free, model-agnostic framework that effectively extends the resolution of diffusion models across both U-Net and DiT architectures. Its key components—Neighborhood Patch Attention, Latent Frequency Mixing, and Structure Guidance—are well-motivated, and shown through extensive experiments to yield competitive or superior results compared to recent baselines while remaining efficient. While some concerns remain about presentation clarity and completeness of comparisons, the reviewers generally agree that the core contributions represent a meaningful advancement for high-resolution image synthesis.
Decision: Accept (poster)