SSIMBaD: Sigma Scaling with SSIM-Guided Balanced Diffusion for AnimeFace Colorization
Authors: Junpyo Seo
HanbinKoo
Jieun Yook
Byung-Ro Moon
Venue: NeurIPS 2025 poster ()
Keywords: Diffusion Models
Noise Scheduling
Perceptual Consistency
Reference-guided Generation
Conditional Diffusion Models
Generative Modeling
Sketch-to-Image Translation
Structural Similarity Index
Primary area: applications
TL;DR: We propose SSIMBaD, a diffusion-based anime sketch colorization method with SSIM-guided noise scaling. It ensures perceptual consistency across steps, leading to better structural fidelity and style transfer than prior models.

Abstract: We propose a novel diffusion-based framework for automatic colorization of Anime-style facial sketches, which preserves the structural fidelity of the input sketch while effectively transferring stylistic attributes from a reference image. Our approach builds upon recent continuous-time diffusion models, but departs from traditional methods that rely on predefined noise schedules, which often fail to maintain perceptual consistency across the generative trajectory. To address this, we introduce SSIMBaD (Sigma Scaling with SSIM-Guided Balanced Diffusion), a sigma-space transformation that ensures linear alignment of perceptual degradation, as measured by structural similarity. This perceptual scaling enforces uniform visual difficulty across timesteps, enabling more balanced and faithful reconstructions. On a large-scale Anime face dataset, SSIMBaD attains state-of-the-art structural fidelity and strong perceptual quality, with robust generalization to diverse styles and structural variations.

SSIMBaD: Sigma Scaling with SSIM-Guided Balanced Diffusion for AnimeFace Colorization
Junpyo Seo Department of Computer Science Scoul National University jsseo99asnu. ac kr
Hanbin Koo Department of Computer Science Scoul National University nagnebin@snu. ac kr
Jieun Yook
Byung-Re Rong-Rooooo Moon' Department Computer Science Seoul National University moon@snu, ac, kr
Department of Computer Science Seoul National University yookje@snu.c ac kr
Abstract
We propose novel diffusion-based framework for automatic colorization of Anime- style facial sketches, which preserves the structural fidelity the input sketch while effectively transferring stylistic attributes from reference image. Our approach builds upon recent continuous-timg time diffusion models, but departs from traditional methods that rely on predefined noise schedules, which often fail to maintain per- ceptual consistency across the generative trajectory. To address this, we introduce SSIMBaD (Sigma Scaling with SSIM-Guided Balanced Diffusion), sigma-space transformation that ensures linear alignment of perceptual degradation, as mea- sured by structural similarity. This perceptual scaling enforces uniform visual difficulty across timesteps, enabling more balanced and faithful reconstructions. On large-scale Anime face dataset, SSIMBaD attains state-of-the-ary structural fidelity and strong perceptual quality, with robust generalization to diverse styles and structural variations. Code and implementation details are available
Introduction
The rapid growth of content industries such as webtoons, animation, and virtual avatars has intensified the demand for automatic generation of high-quality Anime-style images. Among the various sub- tasks, colorizing sketch images remains labor-intensive step in the content creation pipeline, line art lacks shading and color information, requiring significant manual effort from artists. Automating this process not only enhances production efficiency but also ensures visual consistency across frames and styles 7 10].
Early colorization models have been predominantly based on Generative Adversarial Networks (GANs). For instance, 27 B7 leveraged conditional GANs guided by sparse color scribbles as user-provided inputs. However, these methods rely heavily user hints and are sensitive to scribble placement and spatial correspondence. To alleviate this, Lee al. [18] proposed reference-based colorization using Spatially Corresponding Feature Transfer (SCFT) module that extracts semantic correspondences between the sketch and reference images. Yet, their approach struggles under large domain gaps structural mismatches, a challenge that persists across reference-guided generation settings [9]].
"ooresponding author.
htttps://itthbb com http://hhhh...oom//eeeeeaiieee/s -SigmaSaaallnnnggg..... with SSIM Guided Bal anced Diffusion- www.ie/aaae-oooorraaaaaaaaaaa
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Recently, diffusion models have emerged as powerful class of generative models capable of producing
images while avoiding common GAN pitfalls such as mode collapse and In particular, 3 was the first to apply Denoising Diffusion Probabilistic Models 13 to anime face colorization. By leveraging pixel-Ievel supervision and multi-scale structural similarity, they reported improvements in PSNR, MS-SSIM
training instability 9
13
28 30
[3 and FID [22
over GAN baselines. However, their discrete-cosine-best SSIM in early timesteps and then flattens later, yielding uneven perceptual difficulty across the trajectory. This non-uniform degradation complicates reverse-trajectory learning and hinders recovery of fine-grained color textures 31..
forward noise schedule rapidly degrades
Alongside supervised diffusion, a line of exemplar-driven. zero-shot editing and transfer methods has gained traction in the Stable Diffusion (SD) Latent Diffusion Model (LDM) ecosystem [25]. Cross-Image Attention [1. Attention Distillation 39 and StyleAligned [1] exploit attention maps (reflecting semantic correspondences across domains) and feature/statisties modulation (e.g., AdaIN) to perform appearance transfer without task-specific training. While appealing, these approaches are typically tied to large SD/LDM backbones, may require text prompts or additional conditioning, and incur considerable inference latency. In contrast, ControlNet (36) introduces trainable control branches on top of SD and does require fine-tuning but still inherits heavy foundation-model inference. In this work, our goal is orthogonal: we pursue a lightweight and supervised colorization pipeline targeted at near real-time inference and faithful recovery of gegefrreuencc details. We nevertheless include these SD/LDM --bsedd methods and a ControlNet variant in our evaluation for completeness and cross- cross- pooss-aaadigg comparison.
Elucidated Diffusion Models (EDM) 14 introduced continuous-time noise parameterization in o-space, enabling finer control over corruption and improved sample quality. Despite its strengths, EDM's standard (logarithmic) a sampling induces perceptual non-uniformity colorization, where consistent difficulty across the trajectory is crucial.
To address this, we propose SSIMBaD (Sigma Scaling with SSIM-Guided Balanced Diffusion), sigma-space transformation ø² oa)rr that aligns perceptual degradation linearly in SSIM, enforcing uniform visual difficulty across timesteps. Integrating this schedule into the EDM framework yields a continuous o-space diffusion model tailored for Anime-style sketch colorization. The same SSIM- aligned schedule used in training and in a lightweight reverse-only trajectory refinement stage: during training it prevents bias toward either near-clean or extreme-noise regimes; during inference it maintains consistent reconstruction difficulty over steps. Unlike prior methods that optimize reverse fidelity purely empirically, we explicitly anchor both forward and reverse dynamics to SSIM [33]]8] 38). On a large-scale Anime face dataset, this design achieves state-of-the-art structural fidelity while delivering strong, competitive perceptual quality, with robust generalization across diverse styles and structural variations.
Our main contributions are summarized as follows:
A Novel Unified Framework for Perceptually Balanced Diffusion: We propose SSIM- BaD (Sigma Scaling with SSIM-Guide Balanced Diffusion, pioncering framework that balances structural and stylistic fidelity in anime face colorization. Unlike prior approaches that suffer from inconsistencies in perceptual quality, SSIMBaD integrates perceptual sched- ule alignment, training-time consistency, and trajectory refinement to achieve stable and high-quality generation.
A Perceptual Sigma-Space Transformation for Enhanced Stability and Consistency We propose a novel sigma-space transformation, ('()), as a core innovation within SSIMBaD. By linearly aligning SSIM degradation across diffusion timesteps, this perceptual rescaling mechanism significantly improves step-wise consistency during the generation process, ensuring consistent perceptual generation, and overcoming the limitations of conventional noise schedules that often bias towards low or high-frequency details.
State-of-the-Art Performance Validated by Comprehensive Experimentation Exten- sive experiments on Danbooru AnimeFace show that SSIMBaD sets a new state of the art on PSNR, MS-SSIM, and FID, and delivers competitive LPIPS, CLIP-I, and DINOv2-I with strong cross-refereece generalization. Ablations attribute the gains to all components: the EDM backbone, SSIM-aligned sigma scaling, and trajectory refinement.

---

2
Related Works
GAN-Based Sketch Colorization Early colorization models primarily relied GANs, guided by user-provided inputs such as sparse color scribbles 7 27 37. While effective, these approaches are highly sensitive scribble placement and often fail generalize. To mitigate this, Lee et al. [18] proposed reference-basee method using SCFT module, which extracts semantic alignments between sketches and reference images. However, SCFT remains vulnerable to domain gaps and structural mismatches [19]. Other works explored semi-automatic pipelines (10) and two-stage GANs for flat-filling and shading [77, or incorporated text tags for semantic guidance [6]. but challenges in consistency and stability persist.
Generative Diffusion Models Diffusion models have emerged as powerful generative frameworks that address key limitations of GANs, including mode collapse and training instability [R]
29 23 g
26]. By learning to reverse gradual noising process, they enable stable training and high-quality image synthesis. Nichol and Dhariwal 23 demonstrated that well-tuned diffusion models can outperform GANs across diverse benchmarks.
Subsequent advancements have improved their flexibility and performance. Song et al. [30 32] introduced a score-based formulation using stochastic differential equations (SDEs), enabling continuous-time generation and principled control over sampling dynamics. In parallel, several works have proposed deterministic sampling methods based on ordinary differential equations (ODEs), such PNDM (20) and DPM- -ooler [2I]. which accelerate inference while maintaining generation quality. Karras et al. [14] extended this with EDM, which operate in continuous o-space and decouple noise level selection from timestep scheduling. EDM achieves state-of-the-ard results on high-resolution
datasets such as FFHQ 15 and ImageNet [
Seference-Guider Diffusion Colorization Diffusion models have shown strong potential for image colorization by conditioning the denoising process inputs such sketches or reference images. Techniques like classifier guidance 19], cross-ate atteetootn and adaptive normalization enable fine-grained control. User- essr-guided methods such SDEdit [22] and DiffusArt [5] leverage partial noise or scribbles for controllable generation, but often require carefully crafted inputs. ILVR [6] and ControlNet (36] improve precision via reference alignment and auxiliary signals, yet depend
heavy Stable Diffusion / Latent Diffusion backbones 25
Beyond these, SD/LDM-based zero-shot transfer methods Cross-Image Attention (1. Attention Distillation [39], and StyleAligned [[]] proppaaae appearance via cross/shared attention and AdaIN, sometimes with text prompts or inversion, and are training-free but incur notable memory/latency costs. ControlNet, turn, requires fine-tuning on SD while retaining foundation-mods overhead. In contrast, target a lightweight, supervised, jointly conditioned (sketch, reference) diffusion pipeline designed for near real-time inference and faithful gigh-rreuency detail recovery; nevertheless, we include modified ControlNet and the two pero-shot methods as baselines for fair cross pross-paradigm comparison.
AnimeDiffusion Cao et al. [3] propose DDPM-based [13] pipeline for anime face colorization using UNet denoiser with early-fusion conditioning: the sketch, the reference, and noise-level map are channel-wise concatenated at the input rather than fused via cross-image attention or adapter modules. Training employs pixel-wise supervision and the forward corruption follows a discrete- cosine schedule. This raw-piling design simple and efficient, relying on supervision to learn alignment without explicit correspondence layers.
Background: Elucidating the Design Space of Diffusion-Based Generative Models
The EDM framework 14 generalizes DDPM by introducing continuous-time formulation of the forward noising process based on a scale variable E [amin: mmax]. which replaces the discrete timestep index Under this formulation, a clean image xo is perturbed into a noisy observation using a continuous noise level:
3

---

z(o) Io 0
c~.
(I)
This allows the model to learn over continuous spectrum of corruption strengths, offering greater flexibility than DDPM's fixed timestep schedule. For notational simplicity, we denote æ(o) ) as x in the subsequent descriptions.
To stabilize training and ensure scale-invariant learning, the noisy input is preconditioned using the noise level o and fixed constant data (typically 0.5). The network F& takes and as input and produces denoised estimate. The final prediction (2 æ), which serves as function approximator of the clean target 2o, is computed using noise dependee skip connections, as defined by:
Do(æ; o = Cakip(a) - Coun (o) Fo(cino)) x, xr
where Cskip. Cin. and Cour are predefined scaling coefficients derived from a.
At inference time, EDM defines the generative process as a reverse-time probability flow ODE, derived from the SDE framework in score- bosee diffusion models [32]
dx 1
dt
This ODE is numerically integrated using Euler or higher-order methods such as Heun or Runge- Kutta.
To discretize this continuous formulation, EDM introduces a p-parameterized noise schedule:
1//0 )) i= =
1/p max +
Mo
oi
By adjusting sampling steps can be concentrated in Iow- or high-noise regions. Most constants and scheduling heuristics this formulation are directly adopted from the original EDM framework [14].
4 SSIMBad: : Sigma Scaling with SSIM-Guided Balanced Diffusion for AnimeFace Colorization
We propose SSIMBaD, which incorporates a perceptually grounded noise schedule into the EDM 14) Unlike prior log-based schemes, SSIMBaD aligns forward and reverse trajectories using transfor- mation that ensures perceptually uniform SSIM degradation.
The model conditions on Jeumd E Hxxx formed by concatenating a reference image Jeer and sketch Takench The clean target [AI RHW x3 is corrupted with Gaussian noise to produce Tnoises which is denoised over time conditioned on Teond- We now describe the key components of SSIMBaD, while Appendix A further elaborates how sketches and reference images are transformed, well as the data augmentation strategies applied.
4.1 SSIM-aligned Sigma-Space Scaling
The perceptual quality of diffusion models is highly sensitive to how noise is distributed across the denoising trajectory. EDM, inference uses p-parameterized schedule (4) to sample noise levels in nonlinear manner, typically concentrating steps near low-noise regions. In contrast, training samples ln from a log-normal distribution V(Pmean Pla), implicitly assuming a different transformation. This discrepancy implies that the transformation applied during training, Quain (a) log(o), differs from that used in inference, @inference (0) o resuliing in perceptual misalignment between forward and reverse trajectories.
To resolve this, we propose SSIM-aligned sigma-space scaling perceptually motivated strategy that defines shared transformation Φ R2 + R used consistently across both training and inference. This transformation maps the noise scale to perceptual difficulty axis, ensuring visually uniform degradation throughout the diffusion process. Based on this transformation, we construct the noise schedule by interpolating linearly in the dispace:
σ. = --- (d(omin) + N-1 (ø(omaax - (mmm)))) = 0, I....... .. N
(5)

---

To identify the optimal Ø', we consider a diverse candidate set & of analytic and squash-like transformations:
o, (o)( log(1 + o0)
o², o²'
arcsinh(o), tanh(o),
a
ap
(i))
log(o? + 1), arctan(o)
+1' a + op
where 0 and 0 are tunable constants. Each & is evaluated by how linearly its induced noise schedule aligns with perceptual degradation. measured by SSIM. Specifically, we compute the coefficient of determination (R²) between of and SSIM degradation under additive noise:
R22 (1(o7. SSIM
n, Lgz 4a))
arg max E Imm des
(6)
where (Ig,, n) are drawn from the data distribution and Gaussian noise, respectively. We define the coefficient of determination as:
i= (yt y/ Σ (y: 7)
R (((()
where ys the prediction of ys from the best linear fit to (2:) and ýj their sample mean.
Our empirical search reveals that 6 (a) 5+0.3 yields the highest R²² and near-linear SSIM degradation. We adopt this transformation consistently in both training and inference, unifying the sampling dynamics across the diffusion process.
In addition, we replace the conventional log(a) noise embedding with Croise (a) to align temporal conditioning with the perceptual trajectory. This alignment stabilizes training, improves reconstruc- tion fidelity, and enhances generalization across diverse reference domains (see Section 5.3.1).
4.2 Framework of SSIMBaD
The denoising model Dy follows preconditioned residual design adapted from EDM 14) where the noisy input is scaled and fused with learned residual correction. Distinctively, we replace the conventional log(or noise embedding with a perceptually grounded squash function (ooise(O) (a) 0+0.3 ensuring better alignment with visual difficulty across the noise trajectory.
Denoising Network
Formally, the denoiser is defined as:
Do moise Loundi σ) = Cskip (o) mnoise + Cour(a) F6 (Cin(a) Inoiser Lcond's o² (o))
Training To expose the model to perceptually balanced distribution of noise scales, we sample such that (a) ( uniformly distributed over (6" (amin) 1 di mm)ax)] The noise embedding Cnose set to o² (or), replacing traditional log-variance encodings. Given noisy input Lgt + with N(0, I)),, the pretraining loss is:
(8) Trajectory Refinement To further enhance perceptual fidelity, we apply trajectory refinement. The reverse diffusion process is initialized from a pure Gaussian noise sample ((N-1) A/00, I), and integrated backward using a perceptually scaled sigma schedule [a, [o,) W derived from (a). For each denoising step i = N 1...., (0o 0), we perform Fuler updates as:
Liain (gz (Cmma))
(amin) . 10000((0mm (dmax)) Bly-pan Fn- Do + n. Teund; σ) - Ig
((-1) 7()) Als (T(), Temms: σ;
oz) Teums:
r(1) Ad, = 0:-1-
(9)
Instead of treating the entire process as fixed, we fine-tune the denoising trajectory itself by aligning the final reconstruction (0) with the clean target Lgt via an &g loss:
Lurjectory refinement Esprpan En 1:00 181
(10)
5

---

Here, ((0) denotes the reconstructed image obtained after performing the full reverse trajectory from N down O. This refinement can interpreted as fine-tumin fin-ttning step that encourages the entire denoising trajectory to terminate closer to the ground-truth image.
Inference During inference, schedule:
the same x (a) transformation and construct deterministic
o = (x))
(16 (o -000 (A i == 0, N
(11)
N
We then apply the same Heun's methodfollowing the formulation in EDM, in trajectory refinement to produce the final image from pure noise.
Experiments
5.1 Dataset Description
We evaluate our method benchmark dataset introduced by [ specifically curated for reference- guided anime face colorization. The dataset comprises 31, 31,696 6966 sketch-color training pairs and 579 test samples, all resized a resolution of 256 256 pixels. Each training instance consists of ground-truth color image Lat and its corresponding sketch Lseuch. generated via an edge detection operator such XDoG [3]. The sketch images serve as the structural input, while the reference images provide appearance cues such as color and style.
We evaluate model robustness under two test settings. In the same-reference scenario, the reference image is a perturbed version of the ground-trun sharing the same structural input as Iakech- In the cross-reference scenario, the reference is randomly sampled from other test images, introducing variations in both color and facial attributes. This dual setup enables evaluation of reconstruction fidelity under ideal conditions and generalization under domain shift.
5.2 Evaluation Metrics
Fidelity metrics focus on low-level accuracy and structural consistency. PSNR measures pixel-Ievel reconstruction quality via mean squared error, though it correlates weakly with human perception 2 MS-SSIM extends SSIM across multiple scales of luminance, contrast, and structure [34]. making it appropriate for sketch-conditioned colorization. FID computes the Fréchet distance between generated and real image features 112, capturing distributional realism and overall fidelity.
Perceptual metries capture semantic and stylistic alignment beyond pixel fidelity. LPIPS quantifies perceptual distance using deep features [38 reflecting human judgments of texture and style plausi- bility. CLIP-I measures semantic consistency through cosine similarity of CLIP embeddings between generated and reference images 224) DINOv2-I evaluates structural and style-level similarity using
self-supervised visual features 4. offering stable perceptual indicator less rext-biased than CLIP.
5.3
Experimental Results
5.3.1 Empirical Evaluation of SSIM-Aligned Sigma-Space Scaling Functions
Table 1: Transformation functions ((a) sorted by R2 linearity with SSIM degradation. Bounded squash functions yield the highest perceptual alignment.
((a) R² 0.0616
log(o² +
log Ip(a)
arcsinh)())
cP
2
0.0624
0.0768
0.1183
0.2225
0.3754
0.4001
0.7332
((0)
sigmoid(ar)
tanh(ar
log(a)
7+0.0
0+0.7
7+0.1
a+0.3
0+0.3
R²
0.6837
0.8196
0.8650
0.8710
0.8972
0.9275
0.9277
0.9793
To ensure perceptual consistency across the generative trajectory, we construct the noise schedule by uniformly sampling in a transformed ((o) space and applying its inverse. We empirically select

---

(b)
(c)
(d)
(e)
hh)
(k)
Figure 1: Qualitative comparison of colorization results under the same-reference scenario. From left to right: (a) Sketch input. (b) Reference image. (c) SCFT (88. (d) AnimeDiffuusin 3
(pretrained). (e) AnimeDiifuusion [3] (finetuned). (f) AnimeDiffusion (EDM backbone, default
o-schedule). (g) ControlNet [36] (h) Cross-Image Attention [.] (i) Attention Distillation [39] (i)
Our model (w/o trajectory refinement). (k) Our model (w/ trajectory refinement).
(b)
(c)
d
(e)
g
(h)
(k)
Figure 2: Qualitative comparison of colorization results under the cross-reference scenario. From left to right: (a) Sketch input. (b) Reference image. (c) SCFT (18). (d) AnimeDiffusion [
(pretrained). (e) AnimeDiffusion 3 (finetuned). (f)
(EDM backbone, default
a-schedule). (g) ControlNet [36] (h) Cross-Image Attention M. (i) Attention Distillation (39]
Our model (w/o trajectory refinement). (k) Our model (w/ trajectory refinement).
((o) ++0 based on its near-linear SSIM degradation behavior, Full analysis details are provided in Appendix
To construct a perceptually uniform noise schedule, we empirically analyze the relationship between SSIM degradation and transformed noise levels Φ(ar) for various candidate functions. For each transformation é, a clean image Jeean is corrupted at N 50 different noise levels by adding scaled Gaussian noise as defined in
5.3.2 Evaluation under Same and Cross Reference Scenarios
Table together with Figures and supports category- analysis of the evaluation. For clarity, baselines are grouped into four families: GAN-based (SCFT (88), train-free attention (Cross-Image Attention [1 Attention Distillation (9)). light finetuning (ControlNet [36),, and the AnimeDiffusion family (pretrained, finetuned, EDM backbone 3 144) Our method (SSIMBaD) is reported with and without trajectory refinement. This organization makes explicit how architectural biases manifest as distinct trade-offs among structure, realism, and style, and it highlights why SSIMBaD achieves the most favorable overall balance.
The GAN-based model provides a stable classical anchor. SCFT 18 is competitive under the same-reference condition and remains relatively robust under cross-reference, yet it consistently

---

trails SSIMBaD on cross-reference fidelity. Style indicators are moderate. This pattern suggests that stability alone insufficient to dominate when the reference is mismatched.
Table 2: Quantitative comparison under both same-reference and cross-reference settings across fidelity (PSNR, MS-SSIM, FID) and style-aware (LPIPS, CLIP-I, DINOv2-I) metries. Best results per column are in bold, second-best are underlined.
Method
Training
LPPSS
CLIPPT
DINOv2-1
Same
Croos
Same
Cross
Samn
Cross
Samee
Cross
Samr
Crnos
Same
Croo
SCFT 18
300 spocht
17.17
15477
0.7833
0.7627
43.98
45.18
0.1728
0,5008
0.9020
0.8247
0:9392
0.8622
Aifmeliifasion
3
ppettninedi
300 epechs
11.39
11:39
0.6748
0.6721
06.96
46.72
0,2225
0.5107
0.8993
0.8203
0.9992
0.8576
Anifasiiifasiom
spachs
13.32
0.7091
CuntrolNie 54
specchs
14.74
12.08
0.7336
0120077
40.20
50.25
0,2043
0.4030
0.9194
0.311
0.9641
05739
Cross Image Attention
free
13.95
10.60
0.7147
0.4932
53.63
00.54
0.2663
0.456
0.939
0.855
09335
0.873
Amention Distilatioon
SSIMBaD (wls injoctery telinamentt)
300 spodit
15:15
13.04
0.7115
0.6736
53.53
55.18
0.1878
0.4889
0.8975
0.8382
09339
0.865
SSIMRaD (wl rajectory refinmment
303+10 spachs
1892
15.84
0,812
0.8307
37.10
0.1174
0.4804
0.9134
0.8508
0.9644
0.876
Train-free attention methods emphasize semantic transfer via attention mechanisms without task- specific training on our data. As expected, this inductive bias aligns well with style metrics when the reference is aligned, but it does not reliably preserve structure or realism under mismatch. Attention Distillation [39] attains best-in-class same-reference columns for both structure and style, but degrades sharply under cross-reference (for example, MS-SSIM and FID deteriorate markedly), indicating brittle behavior when appearance cues no longer coincide with content. Cross-Image Attention [6)] achieves the lowest cross-reference LPIPS, confirming its semantic alignment bias, but is less competitive on the remaining style indicators and on fidelity.
Light finetuning with ControlNet (66 strengthens several safe-reference columns yet exhibits a clear drop under cross-reference. Style metrics remain decent, which suggests that short finetuning can amplify appearance cues while risking structural drift and realism loss outside the finetuned regime.
Within the AnimeDiffusion family, the pretrained model [3] offers a reasonable baseline. Simple finetuning raises PSNR and, in the same-reference case, MS-SSIM, but severely harms realism as reflected by large FID values, a characteristic addlity-versuu--rallis failure mode. Replacing the discrete backbone with an EDM formulation [14] alone slightly perturbs perceptual alignment in the same-reference regime, which is consistent with the additional optimization burden of a continuous-tinn parameterization. Introducing SSIM-aligned sigma-space scaling on the EDM backbone subsequently recovers and improves structural fidelity, indicating that perceptually paced scheduling is principal driver of reconstruction quality.
SSIMBaD combines an EDM backbone [44, SSIM-aligned sigma-space scaling, and a reverse-only trajectory refinement to deliver the strongest cross-reference fidelity (PSNR, MS-SSIM, FID) while remaining second-best in the same-reference setting; on style and perception stays near the top (LPIPS [38) DINOv2-I (4) with competitive CLIP-I (24), and qualitative results preserve geometry and transfer color without oversaturation. The mechanism is that SSIM-aligned scaling spreads perceptual difficulty evenly across timesteps, yielding steadier training signals and smoother reverse trajectory that a lightweight reverse-only refinement can exploit without overfitting the forward corruption. This accounts for SSIMBalD's cross-reference advantage and consistently high style ranks, and it clarifies the robustness gap with train-free attention methods whose semantic focus lacks explicit structural pacing. In sum, across method families, SSIMBaD offers a balanced and robust solution that is better suited to practical scenarios with reference mismatch than approaches that peak on style but falter on structure or realism.
5.3.3
Comparison of Diffusion Schedules in DDPM, EDM, and EDM with SSIM-Aligned Sigma-Space Scaling
Figure 3 Fillustrates the behavior of the forward diffusion process for a single training image under different noise schedules. Specifically, it plots how SSIM values change across timesteps (N = 25) and visualizes series of 50 corrupted images corresponding to each timestep, allowing intuitive assessment of the degree of corruption. These findings emphasize the erucial role of scheduling in aligning diffusion dynamics with perceptual difficulty.
8

---

(a) SSIM Curve (DDPM)
(b) SSIM Curve (EDM)
(c) SSIM Curve (SSIMBaD)
(d) Noisy Grid (DDPM)
(e) Noisy Grid (EDM)
(f) Noisy Grid (SSIMBaD)
Figure 3: Comparison of forward diffusion schedules. Top: SSIM curves for DDPM (a), EDM (b), and our schedule o (a) (c). Bottom: x 5 corrupted grids (d)-(f) show each schedule's visual effect. Our method yields perceptually uniform degradation across timesteps.
The DDPM baseline employs cosine-based schedule, designed increase noise linearly across discrete timesteps. As seen the graph in Figure (a), DDPM introduces minimal noise during early steps but abruptly escalates noise levels later stages, resulting in uneven SSIM degradation(noise levels) across timesteps. This leads to difficulty in reconstruction during the reverse process.
EDM improves upon DDPM by interpolating noise levels in o-space via a p-parameterized schedule, yielding a smoother degradation curve (Figure (b)). However, SSIM changes are concentrated in the mid-o range, with saturation at both ends. As result, only portion of the schedule contributes effectively to training, reducing overall efficiency and biasing learning toward the central region.
As shown in Figure (c), the proposed (a schedule, which employs SSIM-aligned sigma-space scaling, is designed so that SSIM degradation becomes linear with respect the transformation of The images corresponding to each timestep demonstrate that, at no stage, is there an excessive SSIM degradation; rather, smooth and balanced noise is introduced at every step. This uniformity ensures that all diffusion stages become equally important, thereby improving reconstruction reconstruction fidelity across all frequencies. Furthermore, it enables more stable training and interpretable sampling behavior.
5.4 Ablation Study
Table summarizes cumulative ablation from the AnimeDiffusion baseline to three successive additions: an EDM backbone, SSIM-aligned scaling, and lightweight reverse-only trajectory refinement. Moving EDM raises PSNR in both regimes (same 11.39 to 13.30; cross 11.39 to 12.11) but slightly lowers MS-SSIM and worsens FID, consistent with the higher optimization burden of a continuous-time formulation paired with schedule that is not perceptually paced. Adding SSIM-aligned scaling rebalances per-step corruption by difficulty and recovers structure (PSNR to 15. 15/13.04; MS-SSIM 0.71 15/0.666)))) while LPIPS decreases in both settings; however, FID has not yet improved over the baseline. The reversc-only refinement then converts this headroom

---

into the largest overall gains, yielding the best results across metrics (PSNR 18. 98.92/15.84 MS -SIM 28512/0.8207. FID 34.98/37.10) and pushing style scores near the top (LPIPS 0.1174/0.4804 DINOv2-1 0.9644/0. 00866; competitive CIIP-I). Notably, FID improves by about 12 points in the same-reference case and 10 points in the cross-reference case relative to the baseline, and the gaps between same and cross shrink for MS-SSIM and DINOv2-I, indicating stronger syle-preserving generalization. Overall, the pattern is consistent with a synergistic mechanism: SSIM-aligned scaling equalizes training signal across steps and smooths the reverse trajectory, and small refinement confined to that trajectory delivers simultaneous gains structure and realism, particularly under reference mismatch.
Table 3: Cumulative ablation study under both same- and cross-reference settings across fidelity (PSNR, MS-SSIM, FID) and style-aware (LPIPS, CLIP-I, DINOv2-I) metrics. Each added component incrementally improves model performance across all metrics and settings.
SSIM-aligned
Trajectory
signa-space scaling
Refinement
(t)
MIS-SIM (t)
FID (1)
LPIPS (1)
CLIPAI (
DINOv2-1
Same
Cross
Same
Croes
Same
Cross
Same
Cross
Same
Cross
Same
Cross
46.96
0.2226
0.5107
0.8993
0.8203
0.8576
0.6219
52.18
0.4925
0.8527
0.6736
0.8207
6 Conclusion
This paper presented SSIMBaD, a diffusion framework for anime face colorization that reconciles training and inference with perceptual difficulty. The central contribution is SSIM-aligned sigma- space scaling, which reparameterizes the noise schedule to follow an approximately linear SSIM degradation, yielding uniform perceptual difficulty across steps. Coupled with an EDM backbone and lightweight reverse-only trajectory refinement, the framework aligns forward corruption and reverse reconstruction along single perceptual trajectory.
Comprehensive experiments on the Danbooru AnimeFace dataset validate the approach. Under same- and cross-reference conditions, SSIMBaD attains PSNR 18.92/15.84 dB, MS-SSIM 0.8512/0.8207, and FID = 34.98/37.10, 10, outperforming SCFT, AnimeDiffusion and a modified Con- trolNet baseline (Table 2 Against recent training-free methods, SSIMBal maintains substantially stronger structure and realism under reference mismatch, e.g., cross-reference MS-SSIM 0.8207 and FID 37.10 versus 0.4932/60.55 for Cross-Image Image Attention and 1.5229944 for Attention Distillation. Qualitative comparisons in Figures and corroborate these findings.
Ablation studies (Table show complementary contributions. Moving EDM improves fidelity; SSIM-aligned scaling equalizes per-step difficulty and stabilizes optimization; trajectory refinement then exploits the smoother reverse trajectory to improve realism and semantic coherence. Schedule analysis (Figure confirms that the proposed scaling produces near-linear SSIM decay (high R²),, avoiding early under- and late over-corruption and improving step efficiency. Additional sensitivity analyses indicate robustness to solver choice (Fuler, Heun) and step allocation, and faster convergence under equal compute.
Limitations remain in fine- fine-grained details (c.g., small accessories, iris highlights) under extreme sketch sparsity or large reference-conten gaps. Nevertheless, the empirical evidence indicates that aligning diffusion schedules with perceptual degradation is an effective and general principle. Beyond anime colorization, SSIM-aligned scaling is readily applicable to other conditional generation tasks that require structural preservation and perceptual balance, including sketch-to ceetch-to-immant to-image synthesis and controllable diffusion.
10

---

Acknowledgments
The ICT at Seoul National University provides research facilities for this study.
This research was supported by the Challengeable Future Defense Technology Research and De- velopment Program through the Agency for Defense Development (ADD), funded by the Defense Acquisition Program Administration (DAPA) in 2025 (No. 915110201).
This work was also supported by the BK21 FOUR Intelligence Computing Program (Department of Computer Science and Engineering, Seoul National University), funded by the National Reseurch Foundation of Korea (NRF) (No. 4199990214639).

---

References
[1] Yuval Alaluf, Daniel Garibi, Or Patashnik, Hadar Averbuch-Elow and Daniel Cohen-Or, Cross- image attention for zero-shot appearance transfer. In ACM SIGGRAPH Conference Papers, 2024.
[2] Yochai Blau and Tomer Michaeli. The perception-distoriin tradcoff in image restoration. In Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), 2018.
[3] Yu Cao, Xiangqiao Meng, Pui Yuen Mok, Xueting Liu, Tong- Yee Lee, and Ping Li. Animedif- fusion: Anime face line drawing colorization via diffusion models. IEEE Trans. Vis. Comput. Graphics, 2024.
[4] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Dinov2: Learning robust visual features without supervision. arXiv preprint, arXiv:2304.077 071933, 2023.
[5] Herman Carrillo, Michaël Clément, Aurélie Bugeau, and Edgar Simo-Serra. Diffusart: Enhanc- ing line colorization with conditional diffusion models. In Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition Workshops (CVPRW), 2023.
[6] Jooyoung Choi, Sungwon Kim, Yonghyun Jeong, Youngjune Gwon, and Sungroh Yoon. Ilvr: Conditioning method for denoising diffusion probabilistic models. In Proc. IEEE/CVE Int. Conf. Computer Vision (ICCV), 2021.
[7] Yuanzheng Ci, Xinzhu Ma, Zhihui Wang, Haojie Li, and Zhongxuan Luo. User-guided deep anime line art colorization with conditional adversarial networks. In Proc. 26th ACM Int. Conf. Multimedia (ACM MM), 2018.
[8] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale scale hierarchical image database. In Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pages 248-255, 2009.
[9] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
[10] Chie Furusawa, Kazuyuki Hiroshiba, Keisuke Ogaki, and Yuri Odagiri. Comicolorization: Semi-automatic manga colorization. In ACM SIGGRAPH Asia Technical Briefs, 2017.
[11] Amir Hertz, Andrey Voynov, Shlomi Fruchter, and Daniel Cohen-Os Style aligned image generation via shared attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
[12] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Advances in Neural Information Processing Systems (NeurIPS), volume 30, 2017.
[13] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
[14] Tero Karras, Mika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances Neural Information Processing Systems (NeurIPS), volume 35, pages 26565-26577, 2022.
[15] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), pages 4401-4410, 2019.
[16] Hyunsu Kim, Ho Young Jhoo, Eunhyeok Park, and Sungjoo Yoo. Tag2pix: Line art colorization using text tag with secat and changing loss. In Proc. IEEE/CVF Int. Conf. Computer Vision (ICCV), 2019.
[17] Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. In Advances in Neural Information Processing Systems (NeurIPS), 2021.
12

---

[18] Junsoo Lee, Eungyeup Kim. Yunsung Lee, Dongjun Kim, Jaehyuk Chang, and Jaegul Choo. Reference-basee sketch image colorization using augmented reference and dense semantic correspondence. In Proc. IEEF/CVF Conf. Computer Vision and Pattern Recognition (CVPR), 2020.
[19] Zekun Li, Zhengyang Geng, Zhao Kang, Wenyu Chen, and Yibo Yang. Eliminating gradient conflict in reference-based line-art colorization. In Proc. European Conf. Computer Vision (ECCV), 2022.
[20] Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. Int. Conf. Learning Representations (ICLR), 2022.
[21] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li. and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
[22] Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations. In Advances in Neural Information Processing Systems (NeurIPS), 2022.
[23] Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. CoRR, abs/2102.09672, 2021.
[24] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. Proc. 38th Int. Conf. Machine Learning (ICML), 2021.
[25] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proc. IEEE/CVF Conf. Computer Vision and Patlern Recognition (CVPR), 2022.
[26] Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. In Int. Conf. Learning Representations (ICLR), 2022.
[27] Patsorn Sangkloy, Jingwan Lu, Chen Fang, Fisher Yu, and James Hays. Scribbler: Controlling deep image synthesis with sketch and color. In Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), 2017.
[28] Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In Proc. 32nd Int. Conf. Machine Learning (ICML), 2015.
[29] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Int. Conf. Learning Representations (ICLR), 2021.
[30] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems (NeurIPS), 2019.
[31] Yang Song and Stefano Ermon. Improved techniques for training score-basee bosee-based generative models. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
[32] Yang Song, Jascha Sohl-Dich Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Int. Conf. Learming Representations (ICLR), 2021.
[33] Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli, Image quality assessment: From enror visibility structural similarity. IEEE Trans. Image Processing, 13(4):600-612, 2004.
[34] Zhou Wang. Eero P. Simoncelli, and Alan C. Bovik. Multi-scale structural similarity for image quality assessment. In Proc. Asilomar Conf. Signals, Systems Computers, volume 2, pages 1398-1402. 2003.
13

---

[35] Holger Winnemöller, Johannes E. Kyprianidis, and Sven C. Olsen. Xdog: An extended difference-of -faussssisss compendium including advanced image stylization. Computers & Graphics, 36(6):740-753, 2012.
[36] Lvmin Zhang and Maneesh Agrawala. Adding conditional control to pretrained diffusion models. CoRR, abs/2302.05543, 2023.
[37] Lvmin Zhang, Chengze Li, Tien-Tsin Wong, Yi Ji, and Chunping Liu. Two-stage sketch colorization. ACM Trans. Graph. (Proc. SIGGRAPH Asia), 37:6):61:112611111 2018.
[38] Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreason- able effectiveness of deep features as a perceptual metric. In Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), 2018.
[39] Yang Zhou, Xu Gao, Zichong Chen, and Hui Huang. Attention distillation: A unified approach to visual characteristics transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
14

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justifiiatiin right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
The scope and contributions of the paper are clearly stated in both the abstract and the introduction.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
15

---

Limitations such as difficulties in restoring fine details like eye color are mentioned in the conclusion section.
Justification:
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "I "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [No]
Justification: This paper does not present formal theorems or proofs, but provides empirical evidence.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: The details required to reproduce the main experimental results are provided in Sections 4
[5 as well as Appendices and The code used for the experiments is also available via anonymous link.
16

---

Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g.. a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes)
Justification: The paper explicitly mentions that code and implementation details are publicly
available via anonymized URL provided in the abstract.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ctaips.cc/puulic/guui.
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
17

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: These are fully described in Section and Appendices and
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: The paper presents quantitative comparisons using standard metrics (PSNR, MS- SSIM, FID) without explicitly reporting error bars or statistical significance tests.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula,
call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: All detailed information can be found in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
18

---

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: The paper does not present ethical concerns or violations of the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: Broader impacts such as potential applications in content creation industries and possible generalization to other negative impacts are not extensively detailed.
tasks are mentioned, though specific
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML..
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
19

---

Answer: [NA]
Justification: The paper does not deal with sensitive data or models prone to misuse, thus no specific safeguards are necessary.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes)
Justification:
These are cited properly from the original source.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URI
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, pwpprswithhaeeooo thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-pace re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
No new datasets or significant assets beyond the methodological framework are introduced.
Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
20

---

14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The research does involve crowdsourcing or human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country institution) were obtained?
Answer: [NA]
Justification:
The research does involve human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMss if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The paper does not utilize large language models (LLMs) in its methodological approach.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM policy for what should or should not be described.
21

---

Appendix
A Details of the Proposed Framework
A.1 Conditional Input Construction
Let Lg. E #xxx3 denote the RGB ground-truth anime image, where H and W are the spatial resolution of the image. To form a pair of conditioning signals that guide both structure and style reconstruction, we derive two distinct inputs from Ig:: a structural sketch and a perturbed reference.
The sketch skctch is extracted via the extended Difference- -fuussssinnnnsss (XDoG) operator 35 which enhances edge-like regions through nonlinear contrast enhancement. Formally:
Sketch Extraction
Tsketch (To)( R #xxx1
(12)
This 1-channel sketch preserves eeghffreuueccy structure such contours and character outlines, serving as a strong spatial constraint during generation.
Reference Transformation To simulate reference-guiin generation under diverse style domains, we construct distorted version of zu using sequence of geometric transformations. First, Thin Plate Spline (TPS) deformation is applied introduce local warping, followed by random global rotations to inject non-aligned style cues:
Lef Rotate(TPP((gi)) E x
(13)
This 3-channel reference encodes the target color palette and texture, potentially with mild spatial misalignments.
Channel- Wise Conditioning and reference along the channel dimension:
The final conditional input formed by concatenating the sketch
Ioond
[Ter
Iseuc] R #xwx4
(14)
where denotes channel-wise concatenation. This composite input retains both semantic layout and color style information, enabling the network to model structural consistency and stylization jointly. Note that Tonna held fixed throughout each diffusion trajectory to serve as conditioning context for the denoiser.
A.2 Incorporating EDM
We reformulate B within the continuous-time framework of EDM 14! preserving its U-Net-based conditional denoiser Fe while adopting noise-level parameterization based on continuous scale a rather than a discrete timestep f. This transition from discrete to continuous noise coordinates enables finer-graine modeling of the forward and reverse processes, well improved control over perceptual degradation across the diffusion trajectory.
Under the EDM formulation, the forward process perturbs a ground-truth image Ig. into noisy observation I mosse by adding Gaussian noise of standard deviation a::
[ J N (Inoise: Igt, d²I) Pdata (eood) dIg..
Po (Inoss [cood)
(15)
where Lcond a fixed conditioning tensor (e.g., reference and sketch) and denotes the identity matrix. This parameterization allows the model to operate over a continuous spectrum of noise intensities, removing the timestep discretization bottleneck of DDPM [3]
Noise-Award Preconditioning To stabilize training and normalize feature magnitudes across varying o, EDM applies a noise-aware preconditioning scheme [4]. The denoiser D, is constructed as residual mapping composed of pre-scaled input/output paths:
Dnoosse Leondi σ) Cakip Lnoise + Cour(o) F& Fo (C)CCa(()))) (nises Lcondi Caoise (a)),
(16)

---

where Cin, Cout, and Cakip are scale-dependent coefficients defined as:
data
In
Cskip
Cout
Can
Cnoise
0² +
data
data
0 + o
data
This formulation ensures that input features have consistent scale, preventing signal collapse at low noise amplification at high noise levels. In practice, we use datta = 0.5.
Training Objective Unlike DDPM which samples timesteps E (1...., T), EDM samples In from normal distribution (NPmmam P The training loss is defined over random as:
C = 1n a (P Puess.P
E Pum En- [1 + n, Lond: 0) Lgill
(17)
Sampling via Reverse-Time ODE define a reverse- reverse-time ordinary differentiall equation (ODE) that approximates the likelihood gradient with the denoiser output:
At inference time, EDM uses score-based formulation to
Inere log ([noise Leond: σ) ~ Do moise: Lond σ)
noise
(18)
g²
leading to the continuous reverse-time dynamics:
no
1 (Da(Inoises Leond; σ)) - Inoise)
(19)
dt
Sigma Schedule and Discretization p-parameterized sigma schedule:
To discretize this process, we apply the Euler method using
=
1/p max
N
i
(o 1/p] )], i i 0,
N
(20)
We initialize the trajectory from pure noise I(N-1) NN I) and integrate the ODE in reverse the precomputed {a,} sequence. The denoising step at each index is performed as:
(i-1) -0) At (De(1 Acond: σ.) - 1()), -
(21)
This continuous-timg formulation enables S to benefit from the architectural and sampling improve- ments of EDM, while retaining its original conditioning and loss structure. In Section 4.1 we further extend this pipeline by introducing a perceptual scaling of to ensure uniform SSIM degradation across steps.
B Details on SSIM-Aligned Sigma-Space Scaling
To design a perceptually uniform noise schedule, we empirically analyze the relationship between SSIM degradation and transformed noise levels o(ar) across various candidate functions. For each transformation Φ, a clean image Teleam is corrupted at N = 50 distinct noise levels by adding scaled Gaussian noise described in ( We then compute the SSIM between each noisy image and its clean counterpart to obtain a degradation curve. To quantify the perceptual consistency of each transformation,
plot SSIM values against á(o) and measure the linearity of the resulting curve using the coefficient of determination (1t²). This procedure is applied to 1% of randomly sampled training images, each undergoing 50 corruption steps, yielding comprehensive perceptual degradation profile across a wide range of noise intensities.
As illustrated in Figure 4 plotting SSIM against o(o) reveals that certain transformations induce nearly linear degradation. In particular, bounded squash functions of the form
o(a) σ +
produce the most perceptually uniform trends. Among these, ((0) ++0.3 achieves near-per mear-perfect linearity with R² value of 0.9949. Based on this result, we adopt this transformation our default

---

550M
550M
0.4
0.2
022
0.2
1.0
100
1.0
0.3
0.2
0.5
0.5
9324
500M
950M
0.
2.6
0.0
22
1.2
0.6
2.7
6601 00110
+100
staniol
61:00 2lmma/
8500
200M
04
0.4
0.4
0.2
0.2
Aggto1
tawileo
Figure 4: SSIM degradation across various transformations 6(a). Each curve shows the SSIM between the clean image and its noisy counterpart as the noise level a increases, under a specific transformation ø. The transformations are ordered by increasing linearity (R?). Among them, bounded squash functions of the form ((o) a+c exhibit the most linear degradation trends. In
-
particular, Φ(o) = o+0.3 achieves near-perfect linearity, making well-suited for constructing perceptually uniform sigma schedules. For clarity, we visualize representative subset of the evaluated transformations.
scaling function in sigma-space. Table summarizes the R² values for representative candidate functions.
Finally, we construct our noise schedule by uniformly sampling steps in the transformed é-space and applying the inverse of the selected transformation to compute the corresponding values, as defined in (5). This perceptually aligned schedule ensures that each diffusion step contributes uniformly to structural degradation, which is critical for achieving balanced and stable restoration during generation.
C Extended Qualitative Comparisons
To complement main results, we present qualitative comparisons in both same-reference and cross-reference scenarios (Figures and (6.. In the same-reference scenario, our model produces visually faithful results that align well with both structure and style. In the cross-neference scenario, it generalizes robustly to unseen references, avoiding oversaturation and preserving content. These results highlight the benefit of SSIM-aligned sigma-space scaling and trajectory refinement in achieving perceptually consistent generation.
3

---

C.1 Same-Reference Scenario
(b)
(c)
(d)
(e)
(1)
(g)
(k)
Figure Qualitative comparison under the same-reference scenario. (a) Sketch input. (b)
Reference image. (c) SCFT 18] (d) AnimeDiffusion K (pretrained). (e) AnimeDiffusion R
(finetuned). (f) AnimeDiffusion (EDM backbone, default a-schedule). (g) ControlNet [36].
Cross- Attention [. (i) Attention Distillation [39 (j) Our model (w/o trajectory refinement).
(k) Our model (w/ trajectory refinement).

---

C.2
Cross-Reference Scenario
(b)
(c)
(d)
(e)
(1)
(g)
(k)
Figure 6: Qualitative comparison under the cross-reference scenario. (a) Sketch input. (b)
Reference image. (c) SCFT [8]. (d) AnimeDiffusion [3] (pretrained). (c) AnimeDiffusion 3
(finetuned). (f) Anime
(EDM backbone, default o-schedule). (g) ControlNet (36]. (h)
Imoss-Image Attention T (i) Attention Distillation [39]. (j) Our model (w/o trajectory refinement).
(k) Our model (w/ trajectory refinement).

---

D
Why Did We Add Rotation to TPS?
Table 4: Quantitative results without TPS rotation under both same-reference and cross-reference
settings. Finetuning improves visual fidelity in both conditions.
Method
PSNR 1
MS-SSIM
FID
Same
Cross
Same
Cross
Same
Cross
SSIMBaD (w/o trajectory refinement)
20.55
11.34
0.8446
0.5996
56.18
65.69
SSIMBaD (w/ trajectory refinement)
23.10
14.00
0.9190
0.7714
24.35
40.73
Despite visually plausible results Figure especially after trajectory refinement, Table reveals significant performance gap between same- and cross-reference scenarios. For instance, PSNR drops from 10 dB to 14.00 dB, and MS-SSIM from 0.9190 to 0.7714, 14, highlighting limited referential generalization. To address this, we introduce lightweight affine rotation into the TPS pipeline, improving alignment between the sketch and reference. As shown in Table 2 incorporating TPS rotation reduces the PSNR and MS-SSIM gaps from 9.1 dB and 0.1476 to 3.08 dB and 0.0305, respectively. FID also improves, and our method surpasses all baselines under cross-reference scenario while retaining strong performance in the same-reference scenario.
E
Does SSIM Behave as Intended During Generation?
HOWRTOR loste1ation)
Rosese 220266
0.6
0.7
Newese (Deneratorn Rever to near (o000 7861)
08
(a) Ours (w/o trajectory refinement)
(b) Ours (w/ trajectory refinement)
Figure SSIM vs o(or curves for the same input image under forward (corruption, red) and reverse (generation, blue) processes. Finetuning improves perceptual linearity in certain regions, but quickly saturates due to existing generation dynamics. The model nonetheless maintains an overall perceptually stable trajectory, suggesting potential for further improvement through trajectory-aware objectives.
To visually examine how closely the model's generation aligns with the intended noise schedule, we plot SSIM against "(a)* for both the forward (corruption) and reverse (generation) processes, using the same input image and schedule.
Figure 7 compares this alignment before and after trajectory refinement. In both cases, the forward trajectory (red) shows near-peree pear-perfect linear SSIM degradation, serving as a perceptual baseline. Notably, the reverse trajectory (blue) already exhibits a fair degree of lincarity even before trajectory refinement, suggesting that the model implicitly learns follow the o(o)) path.
Importantly, trajectory refinement does not disrupt this linearity, preserving perceptual consistency while improving sample quality. These results highlight the robustness of our noise schedule and suggest that further improvements may be possible by designing more principled refinement objectives, which leave for future work.

---

(b)
(d)
Figure 8: Comparison under same- and cross-reference scenarios without TPS rotation. (a) Sketch input. (b) Reference image (same style). (c) Reference image (cross style). (d-e) Our model under same-reference scenario (w/o and w/ trajectory refinement, no TPS rotation). (f-g) Our model under cross-reference scenario (w/o and w/ trajectory refinement, no TPS rotation). Even without explicit rotation-based alignment, our model preserves structural integrity and transfers style consistently across reference domains, outperforming baselines in both scenarios.

---

Implementation Details
To ensure rigorous and reproducible comparisons, reimplemented each baseline model under standardized pipeline. All models were trained and evaluated on the same dataset split, using identical image resolution (256 256), batch size (32), and consistent data augmentation strategy.
Hardware environment: : 2x NVIDIA H100 SXM5 GPUs with a 128-core AMD EPYC 9354 CPU and 512GB RAM. Experiments were conducted using PyTorch 2.1.0 with AMP-based mixed- precision training.
Common hyperparameters:
Optimizer: AdamW; Learning rate: x 10 4; Weight decay: 10 10-2
Scheduler: Cosine decay with epoch warmup
Epochs: 300; Batch size: 32; Gradient clipping: max-norm of 1.0
Distributed training via PyTorch Lightning DDP; 64 data loading workers
F1
Pretraining Comparisons
For fair comparison of the pretraining phase, we evaluated models based on their ability learn from distorted reference inputs and produce structure- proservigg reconstructions.
SCFT
18
Dense semantic correspondence based reference transfer model originally designed for exemplar-guided colorization
Adapted to 256 x 256 resolution
Trained from scratch on our dataset with the same optimizer, learning rate schedule, and number of epochs
AnimeDifflusion
Inffusion-based colorization model trained with fixed iDDPM-style 8-schedule L3
Inference conducted using 50 denoising steps with DDIM 229
Official implementation modified for consistent data split and batch size
F22
Finetuning Comparisons
Finetuning Settings:
Strategy: MSE, depending on baseline capability
Inference time steps: 50 (Euler or DDIM sampling for diffusion models)
Finetuning conducted with preloaded pretrained weights on the same hardware
AnimeDifflusin
MSE-based perceptual finetuning with 50-step DDIM inference 29
Reference and sketch inputs preserved; distorted images created via noise--augmentation
SSIMBaD (Ours):
Pretrained with SSIM-aligned (a) schedule for uniform perceptual degradation
Finetuned using MSE loss, with explicit control over 50 step inference trajectory
E3
Evaluation Metrics:
For both stages, we report PSNR, MS-SSIM 34 and FID 12]. All models were evaluated using 50-step sampling, and outputs were resized to 256 x 256 prior to metric computation.


Reviews:
[{"review_id": "1wYxDgjM0e", "reviewer": "Reviewer_mRSp", "summary": "This paper proposes Sigma Scaling with SSIM-Guided Balanced Diffusion (SSIMBaD) for AnimeFace colorization. The key idea is to apply a sigma-space transformation to achieve a linear alignment with the Structural Similarity Index (SSIM), improving consistency during the denoising process across different timesteps. The authors explore several transformation candidates and empirically select the most effective one. Additionally, they introduce a trajectory refinement to enhance perceptual quality. Experimental results demonstrate that SSIMBaD achieves state-of-the-art performance.", "questions": "1. What is the performance when using trajectory refinement without SSIM-aligned sigma-space scaling?\n2. In Table 4 of the appendix, the caption mentions TPS, but the table content is about trajectory refinement. Is this an error?", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strength**\n1.  The sigma-space transformation is a novel and interesting idea that improves consistency in the denoising process. Moreover, experiments support the effectiveness of this idea.\n2. The experiments in this paper are overall comprehensive. The authors provide ablation studies and empirically justify their choice of transformation.\n\n**Weaknesses**\n1. The ablation study does not report performance for the setting with trajectory refinement but *without* SSIM-aligned sigma-space scaling. In Table 3, trajectory refinement contributes the largest performance gain. This result raises my question of whether trajectory refinement is actually the primary contributor. Moreover, this paper does not discuss how trajectories change before and after refinement.\n\n2. This paper’s presentation could be improved. The motivation in the introduction is unclear, and several abbreviations or notations are not defined in the main text. For example,  \"TPS\" is first mentioned in Line 138 but only defined in the appendix (Lines 404–405). Similarly, \"XDoG\" appears in Line 139 without citation or in-text definition, and is only explained in the appendix (Line 399).  Moreover, in Line 148, the terms $P\\text{mean}$ and $P^2\\text{std}$ are used without prior definition.", "was_revised": true, "final_justification": "The rebuttal addresses my concerns. However, the presentation of this paper still needs to improve. Therefore,   I am raising my score to \"borderline accept.\"", "rebuttal": "### **Q1. How does the model perform with only trajectory refinement, without SSIM-aligned sigma scaling?**\n\n**A1.**\nThank you for this insightful question. While we do not explicitly list an ablation of “trajectory refinement without SSIM-based scheduling” in Table 3, this configuration is effectively represented by the **finetuned AnimeDiffusion baseline** (\\[1]).\n\n* **AnimeDiffusion (finetuned)** uses MSE-based trajectory refinement on top of a pretrained model **without** modifying the original noise schedule.\n* **SSIMBaD** uses the **same refinement strategy**, but applies our proposed **SSIM-aligned σ-schedule**.\n\nDespite sharing the same reverse reconstruction mechanism, SSIMBaD achieves **significantly better performance**:\n\n| Model                      | PSNR ↑    | MS-SSIM ↑  | FID ↓     |\n| -------------------------- | --------- | ---------- | --------- |\n| AnimeDiffusion (finetuned) | 13.32     | 0.7001     | 135.12    |\n| SSIMBaD (w/ traj. refine)  | **18.92** | **0.8512** | **34.98** |\n\nThese results suggest that **most of the performance gain arises from the SSIM-aligned schedule**, rather than trajectory refinement alone. This highlights the importance of perceptual alignment across timesteps during the diffusion process.\n\n---\n\n### **Q2. The caption of Appendix Table 4 refers to TPS, but the table appears to focus on trajectory refinement. Is this a mistake?**\n\n**A2.**\nWe appreciate the reviewer’s careful attention. The caption of Appendix Table 4—*“Quantitative results without TPS rotation”*—may appear misleading at first glance, but its intent is to **highlight the effect of disabling TPS rotation** during reference warping.\n\nTo clarify:\n\n* All results in Table 4 are generated **with TPS warping**, but **without TPS rotation**.\n* The comparison is made **with and without trajectory refinement**, under this fixed warping condition.\n* The key goal is to show that **TPS rotation is critical for cross-reference generalization**, and that its absence limits performance even when trajectory refinement is applied.\n\nAs detailed in Appendix D, enabling TPS rotation significantly reduces the **cross-reference performance gap**:\n\n* PSNR gap (same vs. cross) decreases from **9.1dB** → **3.08dB**\n* MS-SSIM gap drops from **0.1476** → **0.0305**\n\nThus, the caption emphasizes the **absence of TPS rotation** as the main condition under investigation. We will revise the caption in the camera-ready version to clarify this intent.\n\n---\n\n### **Q3. Concerns about undefined abbreviations (e.g., TPS, XDoG) and unclear notation (e.g., ϕ)**\n\n**A3.**\nThank you for pointing this out. We agree that all symbols and abbreviations should be clearly introduced. In response:\n\n* **TPS (Thin Plate Spline)** and **XDoG (Extended Difference of Gaussians)** were first used without formal definitions in the main text. These are **fully defined in Appendix A.1 (Lines 399–405)**, but we acknowledge that early clarification is necessary.\n\n  * TPS: A nonlinear geometric transform for local warping\n  * XDoG: An edge-aware operator for sketch extraction\n\n* For notation:\n\n  * Symbols such as \\$\\phi\\_{\\text{train}}(\\sigma)\\$ and \\$\\phi\\_{\\text{infer}}(\\sigma)\\$ denote the training vs. inference transformation of the noise schedule.\n  * This distinction is drawn from EDM \\[6], where mismatch between training/inference trajectories is known to affect performance.\n  * However, these notations appear without formal introduction in Section 4.1, which we will address.\n\n**In the camera-ready version, we will:**\n\n* Introduce definitions of **TPS** and **XDoG** at their first mention (with citations).\n* Define \\$\\phi(\\sigma)\\$ symbols formally in Section 4.1.\n* Insert cross-references to Appendix A.1 for technical readers.\n\nWe believe these edits will improve clarity and ensure the paper remains self-contained. Thank you again for your helpful feedback.\n\n---\n\n### **Additionally**\n\nWe provide additional experiments to validate the effectiveness of each proposed module. The results are organized as follows: (1) main comparisons across models, (2) ablation of individual components, and (3) fidelity-focused comparison against ControlNet.\n\n#### **1. Style-Aware Metric Comparison**\n\nWe evaluate models using **LPIPS**, **CLIP-I**, and **DINO-I (DINOv2)** to assess perceptual, semantic, and structural consistency under both **same-reference** and **cross-reference** conditions.\n\n| Model                       | LPIPS ↓ (Same/Cross) | CLIP-I ↑ (Same/Cross) | DINO-I ↑ (Same/Cross) |\n| --------------------------- | -------------------- | --------------------- | --------------------- |\n| SCFT                        | 0.1728 / 0.5008      | 0.9020 / 0.8247       | 0.9392 / 0.8622       |\n| AnimeDiffusion (pretrained) | 0.2226 / 0.5107      | 0.8993 / 0.8203       | 0.9392 / 0.8576       |\n| AnimeDiffusion (finetuned)  | 0.2242 / 0.5069      | 0.8797 / 0.8012       | 0.9359 / 0.8554       |\n| SSIMBaD (no refinement)     | 0.1878 / 0.4889      | 0.8975 / 0.8332       | 0.9339 / 0.8605       |\n| **SSIMBaD (full model)**    | **0.1174 / 0.4804**  | **0.9334 / 0.8508**   | **0.9644 / 0.8826**   |\n| ControlNet (best: results9) | 0.2043 / 0.4930      | 0.9194 / 0.8311       | 0.9640 / 0.8739       |\n\n> SSIMBaD achieves the best performance across all three metrics, especially in cross-reference generation, indicating strong style-preserving generalization and perceptual coherence.\n\n#### **2. Ablation Study: Progressive Integration of Components**\n\nWe isolate the effects of each component by progressively applying:\n(1) the EDM noise schedule,\n(2) SSIM-aligned σ-scaling, and\n(3) trajectory refinement.\n\n| Configuration                            | LPIPS ↓ (Same/Cross) | CLIP-I ↑ (Same/Cross) | DINO-I ↑ (Same/Cross) |\n| ---------------------------------------- | -------------------- | --------------------- | --------------------- |\n| Base (AnimeDiffusion pretrained)         | 0.2226 / 0.5107      | 0.8993 / 0.8203       | 0.9392 / 0.8576       |\n| + EDM                                    | 0.2192 / 0.4925      | 0.8886 / 0.8300       | 0.9217 / 0.8527       |\n| + SSIM-aligned σ-scaling                 | 0.1878 / 0.4889      | 0.8975 / 0.8332       | 0.9339 / 0.8605       |\n| + Trajectory Refinement (**full model**) | **0.1174 / 0.4804**  | **0.9334 / 0.8508**   | **0.9644 / 0.8826**   |\n\n> Each component contributes progressively to quality. The SSIM-aligned schedule substantially improves perceptual structure, and the refinement step offers a significant additional boost in semantic and stylistic fidelity.\n\n#### **3. ControlNet vs. SSIMBaD: Fidelity Metrics**\n\nTo complement perceptual metrics, we report **FID**, **PSNR**, and **MS-SSIM** for the best ControlNet model and our full model (SSIMBaD).\n\n| Model                       | FID ↓ (Same/Cross) | PSNR ↑ (dB) (Same/Cross) | MS-SSIM ↑ (Same/Cross) |\n| --------------------------- | ------------------ | ------------------------ | ---------------------- |\n| ControlNet (best: results9) | 40.20 / 50.25      | 14.74 / 12.08            | 0.7336 / 0.2007        |\n| **SSIMBaD (full model)**    | **34.98 / 37.10**  | **18.92 / 15.84**        | **0.8512 / 0.8207**    |\n\n> SSIMBaD significantly outperforms ControlNet in all three fidelity metrics, particularly in the cross-reference setting where ControlNet’s MS-SSIM and FID degrade sharply."}, {"review_id": "RbYUA03f2T", "reviewer": "Reviewer_Jxd7", "summary": "The paper introduces SSIM-guided noise schedulers to enforce uniform visual difficulty across timesteps. The proposed method achieves state-of-the-art performance compared to baseline methods.", "questions": "1. The authors need to further clarify the setting of the same-reference scenarios. In Line 200, it is said that the reference image is a perturbed version of the ground-truth. But from Figure 1, the structure is almost the same as the Sketch input. What kind of perturbations are applied to images?\n2. The proposed method should be compared with more methods, i.e., merely comparing with two methods is not enough.\n3. In Table 2, what are the additional 10 epochs used for? What’s the difference between the pretrained one and the finetuned one?\n4. How does SSIMBaD perform if 10 epochs are finetuned on top of SSIMBaD (w/o trajectory refinement)?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Paper Strengths:\n1. It is reasonable to use SSIM-guided noise schedulers to ensure the uniform visual difficulty across timesteps.\n2. The paper is well-written and easy to follow\n3. The paper conducts extensive experiments to validate the effectiveness of the proposed method.\n4. The proposed method outperforms baseline methods.\n\nPaper Weaknesses:\n1. The authors need to further clarify the setting of the same-reference scenarios. In Line 200, it is said that the reference image is a perturbed version of the ground-truth. But from Figure 1, the structure is almost the same as the Sketch input. What kind of perturbations are applied to images?\n2. The proposed method should be compared with more methods, i.e., merely comparing with two methods is not enough.\n3. In Table 2, what are the additional 10 epochs used for? What’s the difference between the pretrained one and the finetuned one?\n4. How does SSIMBaD perform if 10 epochs are finetuned on top of SSIMBaD (w/o trajectory refinement)?", "was_revised": true, "final_justification": "The rebuttal addresses my concerns. The authors did additional experiments to validate the effectiveness of the proposed method. I keep my original rating to accept the paper.", "rebuttal": "### **Q1. Clarification on perturbations in the same-reference scenario. Figure 1 shows high structural similarity.**\n\n**A1.**\nThank you for the question. The **same-reference setup** does not use an unmodified ground-truth image as reference. Instead, we apply **two controlled perturbations** (Appendix A.1, Lines 404–406):\n\n1. **Thin Plate Spline (TPS) warping** – introduces local, nonlinear spatial distortion.\n2. **Affine rotation** – applies a global transformation for misalignment.\n\nThese perturbations ensure the reference image is **misaligned in structure and appearance**, encouraging **style-guided reconstruction** rather than trivial identity mapping. While Figure 1 may show strong structural similarity, color, orientation, and fine-grain alignments differ subtly but significantly.\n\n---\n\n### **Q2. Comparison is limited to two baselines. Broader evaluation is necessary.**\n\n**A2.**\nWe agree that broader evaluation strengthens the validity of our claims. In addition to SCFT and AnimeDiffusion (selected for their compatibility with sketch + reference input), we conducted **expanded comparisons** covering modern baselines, perceptual and stylistic metrics, and ablation studies:\n\n#### **(1) ControlNet (New Baseline)**\n\nWe included ControlNet as a strong diffusion-based baseline, trained for 10 epochs under the same conditioning scheme (sketch + warped reference) as ours. Despite using a powerful Stable Diffusion backbone, ControlNet performs well in same-reference cases but fails to generalize under cross-reference settings:\n\n| Metric        | Same (best) | Cross (best) |\n| ------------- | ----------- | ------------ |\n| **FID ↓**     | 40.20       | 50.25        |\n| **PSNR ↑**    | 14.74       | 12.08        |\n| **MS-SSIM ↑** | 0.7336      | 0.2007       |\n| **LPIPS ↓**   | 0.1988      | 0.4959       |\n| **CLIP-I ↑**  | 0.9123      | 0.8297       |\n\n#### **(2) Style-Aware Metric Evaluation (LPIPS / CLIP-I / DINOv2)**\n\nTo comprehensively assess perceptual quality and stylistic consistency, we evaluated models using **LPIPS**, **CLIP-I**, and **DINOv2-based image similarity**. Our method (SSIMBaD + Trajectory Refinement) consistently achieves **best cross-reference performance**:\n\n| Model                      | LPIPS ↓    | CLIP-I ↑   | DINOv2 ↑   |\n| -------------------------- | ---------- | ---------- | ---------- |\n| SCFT                       | 0.5008     | 0.8247     | 0.8622     |\n| AnimeDiffusion (finetuned) | 0.5069     | 0.8012     | 0.8554     |\n| ControlNet (best run)      | 0.4959     | 0.8297     | 0.8739     |\n| **SSIMBaD (Ours)**         | **0.4804** | **0.8508** | **0.8826** |\n\nThese results show that SSIMBaD excels in **perceptual fidelity, semantic alignment**, and **style-level coherence**, even under the challenging condition of mismatched references.\n\n\n#### **(3) Ablation Study**\n\nWe further validate the contribution of each component via a stepwise ablation:\n\n| Configuration                            | LPIPS ↓    | CLIP-I ↑   | DINOv2 ↑   |\n| ---------------------------------------- | ---------- | ---------- | ---------- |\n| Base (AnimeDiffusion)                    | 0.5107     | 0.8203     | 0.8576     |\n| + EDM                                    | 0.4925     | 0.8300     | 0.8527     |\n| + SSIM-aligned sigma-space scaling       | 0.4889     | 0.8332     | 0.8605     |\n| + Trajectory Refinement (**full model**) | **0.4804** | **0.8508** | **0.8826** |\n\nEach module provides a **consistent and measurable improvement**, with the largest gain in **semantic and style-aware similarity** achieved through **Trajectory Refinement**.\n\n#### **(4) Backbone Generalization (DiT)**\n\nTo demonstrate that our method is **not limited to U-Net**, we applied the SSIM-aligned schedule to a **DiT-based denoiser**. Preliminary results confirm consistent gains in visual quality and training stability. Full results will be included in the camera-ready version.\n\n---\n\n### **Q3. What is the role of 10-epoch fine-tuning in Table 2?**\n\n**A3.**\nThank you for raising this. In Table 2, the difference between pretrained and fine-tuned **AnimeDiffusion** models is:\n\n* **Pretrained:** Standard DDPM model with original β-schedule, no reconstruction optimization.\n* **Finetuned:** Additional 10-epoch optimization using MSE-based loss, as in \\[1].\n\nWhile fine-tuning improves **PSNR and MS-SSIM**, it **worsens FID**, indicating overfitting and reduced diversity. This trade-off motivates our **more principled refinement strategies**, such as SSIM-aligned scheduling and trajectory-based reconstruction.\n\n---\n\n### **Q4. What if SSIMBaD (w/o trajectory refinement) is also fine-tuned for 10 epochs?**\n\n**A4.**\nThis is a great question. In our formulation, **trajectory refinement is equivalent** to the 10-epoch fine-tuning process—but conducted over **deterministic reverse ODE trajectories** using our **SSIM-aligned schedule**.\n\n> So,\n> **SSIMBaD (w/o refinement) + 10-epoch fine-tuning = SSIMBaD (w/ trajectory refinement)**\n\nThis is already reported in Table 2:\n\n| Model                 | PSNR ↑ | MS-SSIM ↑ | FID ↓ |\n| --------------------- | ------ | --------- | ----- |\n| SSIMBaD (no refine)   | 15.15  | 0.7115    | 53.33 |\n| SSIMBaD (with refine) | 18.92  | 0.8512    | 34.98 |\n\nThese results confirm that **trajectory refinement** is not only an optimization step but also an effective means of perceptual improvement under a well-aligned noise schedule."}, {"review_id": "ZRtzCMGAfN", "reviewer": "Reviewer_erVz", "summary": "The paper introduces SSIMBaD, a reference-guided diffusion framework for anime-style face colorization. The core idea is to replace EDM’s log/ρ noise schedule with a “perceptually aligned” σ-schedule obtained through a hand-picked transformation ϕ*(σ)=σ/(σ+0.3) that yields (approximately) linear SSIM degradation over timesteps. The same schedule is used in training, inference, and a lightweight reverse-only “trajectory-refinement” stage. Experiments on the 256² Danbooru AnimeFace dataset show large PSNR/MS-SSIM/FID improvements over SCFT and the AnimeDiffusion baseline.", "questions": "1. The paper currently compares only against SCFT and AnimeDiffusion. Could you add quantitative results for the more recent controllable-diffusion methods mentioned above (e.g., ControlNet, IP-Adapter, T2I-Adapter) so that the reported gains are positioned against the current state of the art?\n\n2. Have you evaluated the proposed SSIM-aligned σ-schedule with a transformer-based denoiser such as DiT (Peebles & Xie, 2022)? If not, please discuss whether you would expect similar improvements and provide any preliminary evidence or rationale.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths\n1. Clear motivation: The mismatch between perceptual difficulty and conventional σ-schedules in colorization is a real issue; the paper articulates it clearly.\n2. Simple, reproducible technique: The proposed rescaling is easy to implement inside any EDM-style pipeline.\n3. Comprehensive ablations: Table 3 isolates the influence of EDM, σ-scaling, and refinement.\n\nWeaknesses\n1. Missing strong baselines – No comparison with modern controllable diffusion systems (ControlNet[1], T2I-Adapter[2], IP-Adapter[3]) that can also ingest sketches + references.\n\n2.  Backbone diversity lacking – SSIMBaD is tested only with a U-Net. No evidence that the SSIM-aligned schedule benefits modern transformer backbones such as DiT[4].\n\n[1]: Zhang, Lvmin, Anyi Rao, and Maneesh Agrawala. \"Adding conditional control to text-to-image diffusion models.\" Proceedings of the IEEE/CVF international conference on computer vision. 2023.\n[2]: Mou, Chong, et al. \"T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models.\" Proceedings of the AAAI conference on artificial intelligence. Vol. 38. No. 5. 2024.\n[3]: Ye, Hu, et al. \"Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models.\" arXiv preprint arXiv:2308.06721 (2023).\n[4]: Peebles, William, and Saining Xie. \"Scalable diffusion models with transformers.\" Proceedings of the IEEE/CVF international conference on computer vision. 2023.", "was_revised": true, "final_justification": "Most of questions I raised have been solved. The comparison with the  modified ControlNet variant shows the SSIMBaD achieves better performances in term of  fidelity and perceptual metrics. And additional experiments provided by authors also show the advantage of SSIMBaD. Thus, I will increase my score.", "rebuttal": "### **Q1. Can you include comparisons against recent controllable diffusion models like ControlNet, IP-Adapter, and T2I-Adapter?**\n\n**A1.**\nThank you for the suggestion. In the main paper, we compared against SCFT \\[14] and AnimeDiffusion \\[1] due to their direct alignment with our task: **sketch-based colorization guided by a warped reference image**, using **dual image inputs**. In contrast, recent controllable diffusion models such as **ControlNet**, **IP-Adapter**, and **T2I-Adapter** are primarily designed for **text-to-image generation** or **unimodal control** and do not natively support **paired sketch + reference conditioning**.\n\nTo address this, we implemented a **modified ControlNet** variant:\n\n* Reference image is encoded as a separate control stream alongside the sketch.\n* Trained for **10 epochs** under the same data, loss, and architecture conditions as SSIMBaD.\n\n| Model                   | Scenario | FID ↓     | PSNR ↑    | MS-SSIM ↑  | LPIPS ↓    | CLIP-I ↑   | **DINO-I\\*** ↑ |\n| ----------------------- | -------- | --------- | --------- | ---------- | ---------- | ---------- | -------------- |\n| ControlNet (ours, 10ep) | Same     | 40.20     | 14.74     | 0.7336     | 0.1988     | 0.9123     | 0.9640         |\n|                         | Cross    | 50.25     | 12.08     | 0.2007     | 0.4959     | 0.8297     | 0.8739         |\n| **SSIMBaD (Ours)**      | Same     | **34.98** | **18.92** | **0.8512** | **0.1174** | **0.9334** | **0.9644**     |\n|                         | Cross    | **37.10** | **15.84** | **0.8207** | **0.4804** | **0.8508** | **0.8826**     |\n\nDINO-I refers to our updated style metric based on DINOv2 features, used consistently throughout all experiments.\n\nDespite its larger backbone, ControlNet lags behind SSIMBaD across all fidelity and perceptual metrics, especially under cross-reference. IP-Adapter and T2I-Adapter require CLIP-based features without spatial alignment, making them unsuitable for our task. However, we see future potential in integrating SSIMBaD with such modules for cross-modal conditioning.\n\n---\n\n### **Q2. Have you evaluated the SSIM-aligned σ-schedule with Transformer-based denoisers (e.g., DiT)?**\n\n**A2.**\nThank you for the excellent suggestion. In our main experiments, we used U-Net-based denoisers to maintain **controlled comparisons and training stability**, especially under limited resources. However, the **SSIM-aligned σ-schedule is architecture-agnostic by design**.\n\nWe believe that **Transformer-based denoisers (e.g., DiT)** can benefit even more from perceptual schedule alignment, as their **global receptive fields** are particularly suited for preserving **semantic consistency** under perceptual degradation.\n\nTo test this, we have started training a **DiT-based model using our SSIM-aligned schedule**. Preliminary results show consistent improvements in **CLIP-I**, **FID**, and **LPIPS**, validating that the benefits of our schedule extend beyond U-Net.\n\nWe will include full experimental results, visualizations, and ablations in the **camera-ready supplementary**. This demonstrates that our method can serve as a **plug-and-play perceptual scheduler** for **modern diffusion backbones**, supporting broader adoption.\n\n---\n\n### **Additionally**\n\nWe provide additional experiments to validate the effectiveness of each proposed module. The results are organized as follows: (1) main comparisons across models, (2) ablation of individual components, and (3) fidelity-focused comparison against ControlNet.\n\n#### **1. Style-Aware Metric Comparison**\n\nWe evaluate models using **LPIPS**, **CLIP-I**, and **DINO-I (DINOv2)** to assess perceptual, semantic, and structural consistency under both **same-reference** and **cross-reference** conditions.\n\n| Model                       | LPIPS ↓ (Same/Cross) | CLIP-I ↑ (Same/Cross) | DINO-I ↑ (Same/Cross) |\n| --------------------------- | -------------------- | --------------------- | --------------------- |\n| SCFT                        | 0.1728 / 0.5008      | 0.9020 / 0.8247       | 0.9392 / 0.8622       |\n| AnimeDiffusion (pretrained) | 0.2226 / 0.5107      | 0.8993 / 0.8203       | 0.9392 / 0.8576       |\n| AnimeDiffusion (finetuned)  | 0.2242 / 0.5069      | 0.8797 / 0.8012       | 0.9359 / 0.8554       |\n| SSIMBaD (no refinement)     | 0.1878 / 0.4889      | 0.8975 / 0.8332       | 0.9339 / 0.8605       |\n| **SSIMBaD (full model)**    | **0.1174 / 0.4804**  | **0.9334 / 0.8508**   | **0.9644 / 0.8826**   |\n| ControlNet (best: results9) | 0.2043 / 0.4930      | 0.9194 / 0.8311       | 0.9640 / 0.8739       |\n\n> SSIMBaD achieves the best performance across all three metrics, especially in cross-reference generation, indicating strong style-preserving generalization and perceptual coherence.\n\n#### **2. Ablation Study: Progressive Integration of Components**\n\nWe isolate the effects of each component by progressively applying:\n(1) the EDM noise schedule,\n(2) SSIM-aligned σ-scaling, and\n(3) trajectory refinement.\n\n| Configuration                            | LPIPS ↓ (Same/Cross) | CLIP-I ↑ (Same/Cross) | DINO-I ↑ (Same/Cross) |\n| ---------------------------------------- | -------------------- | --------------------- | --------------------- |\n| Base (AnimeDiffusion pretrained)         | 0.2226 / 0.5107      | 0.8993 / 0.8203       | 0.9392 / 0.8576       |\n| + EDM                                    | 0.2192 / 0.4925      | 0.8886 / 0.8300       | 0.9217 / 0.8527       |\n| + SSIM-aligned σ-scaling                 | 0.1878 / 0.4889      | 0.8975 / 0.8332       | 0.9339 / 0.8605       |\n| + Trajectory Refinement (**full model**) | **0.1174 / 0.4804**  | **0.9334 / 0.8508**   | **0.9644 / 0.8826**   |\n\n> Each component contributes progressively to quality. The SSIM-aligned schedule substantially improves perceptual structure, and the refinement step offers a significant additional boost in semantic and stylistic fidelity.\n\n#### **3. ControlNet vs. SSIMBaD: Fidelity Metrics**\n\nTo complement perceptual metrics, we report **FID**, **PSNR**, and **MS-SSIM** for the best ControlNet model and our full model (SSIMBaD).\n\n| Model                       | FID ↓ (Same/Cross) | PSNR ↑ (dB) (Same/Cross) | MS-SSIM ↑ (Same/Cross) |\n| --------------------------- | ------------------ | ------------------------ | ---------------------- |\n| ControlNet (best: results9) | 40.20 / 50.25      | 14.74 / 12.08            | 0.7336 / 0.2007        |\n| **SSIMBaD (full model)**    | **34.98 / 37.10**  | **18.92 / 15.84**        | **0.8512 / 0.8207**    |\n\n> SSIMBaD significantly outperforms ControlNet in all three fidelity metrics, particularly in the cross-reference setting where ControlNet’s MS-SSIM and FID degrade sharply."}, {"review_id": "U31h88SKiU", "reviewer": "Reviewer_CgiV", "summary": "This paper proposed a EDM-based method, namely SSIMBaD, for anime face colorization. The authors believe that a diffusion process that allows the SSIM metric to degrade linearly over time is conducive to training a model with good generative capabilities. To realize this objective, SSIMBaD introduced a $\\sigma$ transformation to noise schedule and achieved symmetry in the training process and testing process. Qualitative and quantitative experiments show that SSIMBaD outperforms the compared methods.", "questions": "See weaknesses and the followings:\n1. Why choose SSIM as a metric for measuring image degradation rather than other metrics such as CLIP-I or FID?\n2. The author emphasizes linear degradation. Why not consider rectified flow instead of using EDM?\n3. Can authors provide the distribution of timesteps after $\\sigma$ transformation?", "limitations": "Although the author explains that SSIMBaD has limitations in handling eye colors, further detailed discussion of these limitations is possible. For example, why has the proposed method not been applied to broader example-based style transfer tasks?", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The paper is organized well  and easy to understand.\n2. The performance of the proposed SSIMBaD is better than the compared methods.\n3. The paper is easy to reproduce and the code is provided.\n\nWeakness:\n1. The authors claim that linear degradation in diffusion process with SSIM metrics can improve the model’s capacity, but there is not enough corresponding experiments to demonstrate this claim. Specifically, the strategy in e.q. (6)  to select the $\\sigma$ transformation only show the coefficient of the linear relationship but not reflect the final metrics, which is not able to prove the superiority of the selected function.\n2. Lack of exemplar-based style transfer methods for comparisons. The anime face colorization problem can be regard a special cases of style transfer. Besides, SSIMBaD requires 2 H100 for training and training-free methods [1-3] only require one for inference. Therefore, it is necessary to compare these methods.\n3. The results presented in qualitative and quantitative experiments are not convincing enough. \n\nQualitative analysis: In addition to the eye color defects already pointed out in the article, the hair in the results generated in Figures 1 and 2 also lost the ‘reflective’ characteristics of the reference image. Another serious issue is that, as shown in the second row of Figure 2, there is a noticeable difference in hair color between the generated image and the reference image, and the reference image does not seem to serve as a guide for the generation.\n\nQuantitation analysis: Although the metrics of SSIMBaD in Table 2 are superior to other methods, they are still at a relatively low level, such as PSNR of 18.92 and 15.84. This may limit the practical application of the model. In addition, there is a lack of corresponding style metrics (e.g., DINO-I) to reflect the style differences between the generated images and the reference images.\n4. Minor weakness: \n(a) Lines 158-159 and e.q. (6) are missing punctuation marks.\n(b) The sub-figures (d-f) had better be resize to the same resolution.\n\n[1] Alaluf, Yuval, et al. \"Cross-image attention for zero-shot appearance transfer.\" ACM SIGGRAPH 2024 Conference Papers. 2024.\n[2] Zhou, Yang, et al. \"Attention distillation: A unified approach to visual characteristics transfer.\" Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.\n[3] Hertz, Amir, et al. \"Style aligned image generation via shared attention.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.", "was_revised": true, "final_justification": "I carefully reviewed the authors’ response and believe they have addressed my concerns well. Many additional experiments provide convincing evidence, so I have raised my score.", "rebuttal": "### **Q1. Why did you choose SSIM as the degradation measure? Were CLIP-I or FID considered?**\n\n**A1.**\nWe chose **SSIM** as the degradation measure to construct a **perceptually uniform noise schedule** aligned with human visual degradation. Unlike L2 or FID, **SSIM is structure-aware, normalized, and differentiable**, which makes it a good proxy for controlling how noise affects image structures over time.\n\nSpecifically, our **SSIM-aligned $\\sigma$-schedule** ensures a **linear SSIM decay** across diffusion timesteps (see Section 4.1, Eq.6). This perceptual alignment reduces the mismatch between forward and reverse processes, improving training stability and sample quality.\n\nHowever, **SSIM alone does not fully capture high-level semantics or style**. To complement this, we evaluated models using **LPIPS**, **CLIP-I**, and **DINO-I**, which are widely used style and perceptual metrics. In particular, we adopt a **refined DINO-I metric** based on DINOv2, which offers strong alignment with human judgment of image-level consistency.\n\nBelow are the **main style metric results**, where SSIMBaD consistently achieves the best scores in both **same-reference** and **cross-reference** conditions:\n\n#### Style Metric Results (Higher is better)\n\n| Model                       | Same Score ↑ | Cross Score ↑ |\n| --------------------------- | ------------ | ------------ |\n| SSIMBaD (traj. refine X)    | 0.9339       | 0.8605       |\n| SCFT                        | 0.9392       | 0.8622       |\n| AnimeDiffusion (finetuned)  | 0.9359       | 0.8554       |\n| SSIMBaD (traj. refine O)    | **0.9644**   | **0.8826**   |\n| AnimeDiffusion (pretrained) | 0.9392       | 0.8576       |\n| ControlNet (best)           | 0.9640       | 0.8739       |\n\n\\* Metric = updated **DINO-I** based on DINOv2 features.\n\nThese results validate that **SSIMBaD outperforms all baselines**, especially in the more challenging **cross-reference generation**, demonstrating not only perceptual but also **style-consistent generalization**.\n\n---\n\n### **Q2. Why was Rectified Flow not used instead of EDM?**\n\n**A2.**\nWe appreciate the reviewer’s suggestion. While **Rectified Flow** offers geometric advantages and sampling efficiency, it does **not natively support perceptually grounded schedule design** or trajectory refinement—the two core components of our framework.\n\nIn contrast, **EDM’s continuous ODE formulation** enables **bidirectional perceptual alignment** and allows us to inject **SSIM-aware degradation** into the generative path.\n\nThat said, we consider integrating our SSIM-based schedule with **Rectified Flow** a **promising direction**, and we plan to explore this as future work.\n\n---\n\n### **Q3. Can you visualize the timestep distribution after applying your \\$\\sigma\\$ transformation?**\n\n**A3.**\nYes. Figure 4 and Appendix B of our submission provide visualizations of **SSIM degradation curves** induced by various \\$\\phi(\\sigma)\\$ transformations.\n\nOur proposed mapping:\n\n$$\n\\phi(\\sigma) = \\frac{\\sigma}{\\sigma + 0.3}\n$$\n\nproduces the **most linear SSIM trajectory**, which promotes **uniform perceptual degradation** over timesteps. This leads to more balanced reconstruction and better sample consistency.\n\n---\n\n### **Q4. The link between linear SSIM degradation and final performance lacks empirical validation.**\n\n**A4.**\nWe agree that empirical correlation between **linearity and performance** is important. While we initially used **\\$R^2\\$ alignment between SSIM and \\$\\phi(\\sigma)\\$** as a proxy (see Table 1 / Fig. 4), we are extending our validation via ablation.\n\nIn the **camera-ready version**, we will include:\n\n* **Multiple σ-transformations**: \\$\\log(\\sigma)\\$, \\$\\tanh(\\sigma)\\$, \\$\\sigma/(\\sigma+c)\\$.\n* **Controlled training**: Identical model, data, and loss, with only the schedule changed.\n* **Metrics**: PSNR, MS-SSIM, FID, LPIPS, CLIP-I, DINO-I.\n\n**Preliminary results** indicate that schedules with lower \\$R^2\\$ (i.e., less perceptual linearity) yield **unstable training and degraded generalization**, particularly in cross-reference settings. This supports our core hypothesis that **perceptual schedule linearity is critical for high-quality generation**.\n\n---\n\n### **Q5. The paper lacks comparison with prior style transfer-based models.**\n\n**A5.**\nThank you for highlighting this valuable point. Style transfer methods such as **StyleCLIP**, **Zhou et al.**, and **Hertz et al.** are primarily designed for **inference-only, unconstrained domains** (e.g., face editing, full-body appearance transfer), and typically do not support **paired sketch + reference input** or training-based colorization.\n\nNevertheless, we agree that contextualizing SSIMBaD against such baselines can strengthen our contribution. To that end, we will include the following in the **camera-ready supplementary material**:\n\n* Adapted pipelines using **public implementations**, modified to accept sketch + reference input.\n* Evaluation under **zero-shot or light adaptation** settings on the same dataset.\n* Reporting of **FID, CLIP-I, LPIPS**, and **qualitative comparisons**.\n\nThis extended analysis will show that **SSIMBaD, despite being designed for supervised diffusion**, performs competitively with or outperforms dedicated style transfer models in terms of **style consistency and perceptual quality**.\n\n---\n\n### **Q6. Qualitative results fail to reflect high-frequency cues (e.g., hair gloss in Fig. 1/2).**\n\n**A6.**\nWe acknowledge that in some examples, fine-grained visual cues—such as **hair gloss, reflections, or specular highlights**—are not fully reproduced. This stems from two key factors:\n\n1. **Sketch-conditioned signal loss**: Sketch inputs lack high-frequency detail, limiting the model’s ability to infer such cues purely from line art.\n2. **Style abstraction in cross-reference**: The model treats the reference image as a **style prior** rather than a pixel target. This avoids overfitting to out-of-distribution references but may suppress stylistic nuances.\n\nTo clarify these limitations, we will:\n\n* Add **annotated captions** in Figures 1 and 2.\n* Include **complementary visualizations** in the supplementary material that highlight when and why high-frequency cues are omitted.\n\nThese additions will help readers interpret the model's qualitative behavior with greater transparency.\n\n---\n\n### **Q7. Absolute PSNR values are low; style similarity metrics are missing.**\n\n**A7.**\nWe acknowledge that the absolute values of PSNR and MS-SSIM may appear relatively low. This is expected, as the goal of the colorization task prioritizes perceptual coherence and style transfer quality over pixel-wise reconstruction. Therefore, in quantitative evaluation, it is important to consider not only absolute scores but also relative performance against baselines and qualitative perceptual quality.\n\nTo better assess perceptual and semantic quality, we report three complementary **style-aware metrics**:\n\n* **LPIPS** for perceptual distance\n* **CLIP-I** for semantic alignment\n* **DINO-I\\*** for structural style similarity (DINOv2-based)\n\n\n#### **1. Style Metric Comparison (LPIPS / CLIP-I / DINO-I, Same / Cross)**\n\n| Model                       | LPIPS ↓             | CLIP-I ↑            | DINO-I\\* ↑          |\n| --------------------------- | ------------------- | ------------------- | ------------------- |\n| SCFT                        | 0.1728 / 0.5008     | 0.9020 / 0.8247     | 0.9392 / 0.8622     |\n| AnimeDiffusion (pretrained) | 0.2226 / 0.5107     | 0.8993 / 0.8203     | 0.9392 / 0.8576     |\n| AnimeDiffusion (finetuned)  | 0.2242 / 0.5069     | 0.8797 / 0.8012     | 0.9359 / 0.8554     |\n| ControlNet (best: #9)       | 0.2043 / 0.4930     | 0.9194 / 0.8311     | 0.9640 / 0.8739     |\n| SSIMBaD (no refinement)     | 0.1878 / 0.4889     | 0.8975 / 0.8332     | 0.9339 / 0.8605     |\n| **SSIMBaD (full model)**    | **0.1174 / 0.4804** | **0.9334 / 0.8508** | **0.9644 / 0.8826** |\n\n#### **2. Ablation Study (Same / Cross, LPIPS ↓ / CLIP-I ↑ / DINO-I ↑)**\n\n| Configuration                            | LPIPS ↓             | CLIP-I ↑            | DINO-I* ↑          |\n| ---------------------------------------- | ------------------- | ------------------- | ------------------- |\n| Base (AnimeDiffusion pretrained)         | 0.2226 / 0.5107     | 0.8993 / 0.8203     | 0.9392 / 0.8576     |\n| + EDM schedule                           | 0.2192 / 0.4925     | 0.8886 / 0.8300     | 0.9217 / 0.8527     |\n| + SSIM-aligned σ-scaling                 | 0.1878 / 0.4889     | 0.8975 / 0.8332     | 0.9339 / 0.8605     |\n| + Trajectory refinement (**full model**) | **0.1174 / 0.4804** | **0.9334 / 0.8508** | **0.9644 / 0.8826** |\n\n> The ablation shows that both the **SSIM-aligned schedule** and **trajectory refinement** progressively improve **perceptual, semantic, and structural similarity**. The refinement step provides the largest gain.\n\n#### **3. ControlNet Comparison (Quantitative Evaluation)**\n\n| Model           | Scenario | FID ↓     | PSNR ↑    | MS-SSIM ↑  | LPIPS ↓    | CLIP-I ↑   | DINO-I\\* ↑ |\n| --------------- | -------- | --------- | --------- | ---------- | ---------- | ---------- | ---------- |\n| ControlNet (best of 10 epoch runs) | Same     | 40.20     | 14.74     | 0.7336     | 0.2043     | 0.9194     | 0.9640     |\n|                 | Cross    | 50.25     | 12.08     | 0.2007     | 0.4930     | 0.8311     | 0.8739     |\n| **SSIMBaD**     | Same     | **34.98** | **18.92** | **0.8512** | **0.1174** | **0.9334** | **0.9644** |\n|                 | Cross    | **37.10** | **15.84** | **0.8207** | **0.4804** | **0.8508** | **0.8826** |\n\n\n#### **4. Backbone Generalization**\n\nWe applied our **SSIM-aligned σ-schedule** to a **DiT-based denoiser**, confirming that the method generalizes well beyond U-Net. Early results show improvements in perceptual and style quality, to be included in the **camera-ready supplementary**."}]

Author rebuttal:

Meta-review: The paper received three borderline accept and 1 accept ratings. Reviewers were concerned about the presentation quality, experiment evaluation of the paper. The authors provided a rebuttal, which addressed most of those concerns.  Two reviewers raised their score to borderline accept. There are still some concerns about the presentation. AC recommends to accept the paper. Please update reviewers' suggestions in the final version of the paper.
Decision: Accept (poster)