Text to Sketch Generation with Multi-Styles
Authors: Tengjie Li
Shikui Tu
Lei Xu
Venue: NeurIPS 2025 poster ()
Keywords: sketch genration
multi-style generation
text to image
diffusion model
Primary area: applications
TL;DR: We propose a novel framework for multi-style sketch generation

Abstract: Recent advances in vision-language models have facilitated progress in sketch generation. However, existing specialized methods primarily focus on generic synthesis and lack mechanisms for precise control over sketch styles. In this work, we propose a training-free framework based on diffusion models that enables explicit style guidance via textual prompts and referenced style sketches. Unlike previous style transfer methods that overwrite key and value matrices in self-attention, we incorporate the reference features as auxiliary information with linear smoothing and leverage a style-content guidance mechanism. This design effectively reduces content leakage from reference sketches and enhances synthesis quality, especially in cases with low structural similarity between reference and target sketches. Furthermore, we extend our framework to support controllable multi-style generation by integrating features from multiple reference sketches, coordinated via a joint AdaIN module. Extensive experiments demonstrate that our approach achieves high-quality sketch generation with accurate style alignment and improved flexibility in style control. The official implementation of M3S is available at  https://github.com/CMACH508/M3S.

Text to Sketch Generation with Multi-Styles
Tengjie Li Shikui Tu' Lei Xu
School of Computer Science, Shanghai Jiao Tong University Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ), Guangdong, China (765127364, tushikui, 1eixu/6sjtu. eew.ejt..edd.... edu. cn
Styles
sketch graduale degree gown
Sydney Opera House
sketch balloon high over the mountains
riding horse
performing kung
flwer
0.0
0.25
0.75
110.0
sketch corgi surfing
Figure 1: Top: Synthesized sketches from a specific-style exemplar by our proposed method. Bottom: Multi-style sketches generated by framework. is used to control style tendency. As y increases, the result's style becomes more aligned with the referenced style 2, and vice versa.
Abstract
Recent advances in vision-language models have facilitated progress in sketch generation. However, existing specialized methods primarily focus on generic synthesis and lack mechanisms for precise control over sketch styles. In this work, we propose a training- free framework based on diffusion models that enables explicit style guidance via textual prompts and referenced style sketches. Unlike previous style transfer methods that overwrite key and value matrices in self- attention, incorporate the reference features auxiliary information with linear smoothing and leverage a style-content guidance mechanism. This design
"Correspondence authors are Shikui and Lei Xu.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

effectively reduces content leakage from reference sketches and enhances synthesis quality, especially in cases with low structural similarity between reference and target sketches. Furthermore, we extend our framework to support controllable multi-style generation by integrating features from multiple reference sketches, coordinated via a joint AdaIN module. Extensive experiments demonstrate that our approach achieves high-quality sketch generation with accurate style alignment and improved flexibility in style control. The official implementation of M3S is available at https::/ htt///gttuut www..h..oo....e..eoee.......eeee CMACH508/M33
Introduction
Sketching, as a universal visual medium with historical roots spanning millennia, demonstrates remarkable accessibility by transcending age and cultural barriers [54]. This unique capacity to convey complex concepts through minimal strokes enables effective pross-linguistic communication,
establishing it as potent tool for ideation and conceptualization. Through historical development, sketching has evolved into a multidisciplinary practice permeating diverse domains ranging from industrial prototyping and artistic expression to educational visualization and recreational applications. This stylistic diversity, amplified by modern digital tools, introduces significant challenges for automated sketch generation systems to capture and reproduce specific artistic styles accurately.
fundamental challenge in advancing sketch generation methods originates from the inherent difficulties in data acquisition. Unlike natural images that can be readily obtained through web scraping, high-quality sketch datasets necessitate specialized artistic expertise and substantial time investment for creation, resulting in constrained dataset scales (37 50 [5]. While freehand sketches with rough strokes exhibit greater accessibility 28 35 including large-scale collections like QuickDraw [1I], their abstract and sparse representations pose inherent limitations for training models to generate high-fidelity sketches with fine-grained details 4 [58). Furthermore, this representational gap significantly impedes the effectiveness of captioning models [22] in producing accurate textual descriptions for sketches, thereby creating a critical bottleneck for text-conditioned sketch generation.
Recent work has explored leveraging pretrained vision-language models for sketch generation. CLIPasso 46) utilizes CLIP's cross-modal embedding space [32] optimize Bézier curve parameters by minimizing semantic consistency loss between rasterized sketches and natural images. Building on this idea, CLIPascene 445] introduces explicit scene decomposition to separate foreground and background elements. However, both methods depend heavily on existing image references for supervision. DiffSketcher (53) takes a step further by incorporating text-driven guidance. combines Stable Diffusion 33 with Bézier curve optimization through score distillation sampling, enabling free-form sketch synthesis from textual prompts. Despite this advancement, existing methods [53 [8] suffer from limited control over stylistic attributes. In particular, text-based style conditioning often lacks the expressiveness and specificity needed to capture fine-grained visual traits, making it difficult to match exemplar styles precisely.
In exemplar- based style transfer, a prominent technical direction involves injecting visual features through the diffusion model's 's self-attention mechanisms. Representative works [ [ 5 propose swapping the K/V matrices derived from the reference images' denoising process into target generation. However, this direct replacement strategy proves suboptimal when handling cross-domain scenarios: The inherent discrepancy between reference and target domains induces misalignment between generated queries Q and substituted K/V features []], leading to both content leakage and deteriorated generation quality. (63] addresses this by introducing attention distillation to align target K/V features with reference characteristics. However, this approach still suffers from excessive feature alignment that compromises output authenticity.
We propose Multi-Style Sketch Synthesis (M3S), training-free framework for generating sketches with diverse and controllable styles. To balance stylistic fidelity and content preservation, we introduce K/V injection scheme composed of three components: (1) Hybrid attention fusion, which injects reference style features (Kref/Vref) with target features (Ktar/Varr) in self-attention layers to integrate style while retaining content semantics; (2) Linear feature blending to mitigate content leakage from the reference; and (3) Separated guidance control, which divides classifier-free free sampling into style and content directions, allowing flexible trade-offs between expression and

---

structure. This coordinated design enables high-quality, style-specific synthesis, as illustrated in the upper portion of Fig. !
We further investigate the feasibility of multi-style sketch generation. Sketch stylistic expression fun- damentally manifests through stroke geometry and sparse texture patterns, contrasting with the dense pixel representations of natural images. Capitalizing on this characteristic, we extend M3S's ' feature injection multi-style synthesis by integrating K| features from diverse references. Additionally, drawing inspiration from I we implement a joint AdaIN modulation on intermediate denoising steps to regulate stylistic dominance. This allows users adjust the blending weights 1) between styles, for example, controlling the clown texture density in the second last row of Fig. [
In summary, our main contributions for multi-style sketch generation. Key and value features from the referenced style sketches are considered auxiliary information, combined with linear smoothing and style-content guidance mechanism, enabling the balance between style consistency and fidelity. (2) We investigate the potential of leveraging pre-trained diffusion models' prior knowledge for sketch generation with mixed styles. By introducing joint AdaIN modulation mechanism, our approach provides users with flexible control over style generation tendencies. (3) We implement our method on Stable Diffusion v1.5 [33] and SDXL [29], with extensive experiments demonstrating the method's effectiveness, outperforming state-of of-thhe-rttttt approaches.
follows: (1) We propose training- free framework
2 Related Work
Sketch Synthesis The field of sketch generation has evolved through distinct methodological paradigms. Early breakthroughs like SketchRNN [[]] established sequential modeling using RNN- based encoder-decoderss for single-category sketch synthesis. Subsequent work enhanced multi-
category generation through structured representations: pixel-space modeling 660 25 41] graph-
based architectures 43 [9], and stroke-level analysis [24] Recent diffusion-based approaches
[4 G improved output quality but remain constrained to coarse category-level control. Although
SketchAgent [47] exploits the powerful text comprehension capability of Large Language Models (LLMs) to guide LLMs to generate sketches by using drawing rules and reference drawing sequences as prompts, the generalizability of such approaches still needs to be further explored. Parallel image-to-sketch extraction research progressed from CNN-based methods [7 23 (requiring paired
training data) to training-fret free techniques leveraging vision- lsiounlangguge priors [46 45 53 551 While
existing solutions address specific aspects of sketch generation, they exhibit critical limitations in simultaneous style-text controllability.
Image Style Transfer Style transfer has evolved significantly since the seminal work of Gatys et al. [10], who first demonstrated neural style transfer using Gram bbsted feature statistics from pre- trained CNNs. Subsequent approaches improved efficiency by replacing iterative optimization with feed-forward networks [19] 44), while AdaIN [13] and WCT (26) enabled real-time arbitrary style transfer through feature statistics alignment. However, these frameworks 2 g are less generalizable for arbitrary style transfer. Different interesting methods are proposed with the progress of the text-to- (22 inverses painting into corresponding textual embeddings to guide the text-to-image generative model in creating images of specific artistic appearance. B-LoRA (9] and DEADiff (30] achieve style-content separation by LoRA weight optimization and joint text-image cross-attention layers, respectively. IP-adapter (56) trained a light-weight network to incorporate style embeddings via cross-attention mechanism. InstantStyle (48) improves the performance by injecting style features into the selective layers of the denoising UNet. CSGO [52] enhances the capacity of the style adapter by training on a curated style dataset. Based on CSGO, StyleStudio 21 proposes the cross- cross-modal AdaIN for harmonizing style and text features. Another method is to perform K/ K/V feature injection [1,5] or distillation [63] of the reference image at the self-attention layers. Notably, StyleAligned [2] also leverages referenced features as auxiliary information, similar to M3S, it differs in methodology. Their method enforces distributional alignment between target and referenced features through statistical constraints. Such rigid matching can degrade performance and cause content leakage when handling sketches with large structural variance (see Section 4.
image diffusion model. InST
3

---

1rv/3
Null bood
C
Self Cel-AMmentionO Kan
inversion
fies
Refarinced
Rior
Scoiee/D Dot. ProdizABenta Bunion
sketch the
Mior
Kiur
Rin
Rich
Corment
card AdSIN Moddue
Vier
Vise
Profs
f2
Null lest Inverion
C
Seff-AmentoonCi. Baate Mlas
Fints
Referenced
Figure 2: Pipeline of the proposed M3S. Given the referenced style sketches Leef and rrefa we invert the two images into the latent space, resulting in latents z,-/1 and z z/=/a The referenced K/V features are extracted from these latents and employed as auxiliary information self-attention layers (Section 3.1) for generating target images tar- A style-content guidance (Section (.3) is applied to balance the fidelity and style consistency. We apply a joint AdaIN module to control the style tendency (Section 3.2). Generating single style sketch is special case in the figure, i.e., blocking out the top or bottom branches.
Methodology
M3S is training-fres framework for multi-style sketch synthesis that integrates textual prompts with reference style sketches through a pre-trained diffusion backbone. As illustrated in Fig. 2 our method supports single- and multi-style generation by blending style features. The single-style scenario is achieved by disabling one of the branches (lower path in Fig. 2 and setting joint AdaIN coefficients " 1. Below, we detail the core components and additional counter-based regular guidance for sparse and abstract freehand sketches.
3.1 Style Features Injection
Given referenced sketches Leef, and Irefa. our framework aims to synthesize the target sketch lar through controlled feature injection. We first formalize single-style generation by integrating reference features into the denoising steps. At each timestep E (T...., 0). we extract key-value pairs (Krefi Vrefi). ). from the self-attention layer of the reference sketch's denoising path, while computing target features (Qtar, Ktor, Vtar) in the corresponding layer. The standard attention mechanism operates as:
Qtar KT tor Vear. ddk
Attention(Quan Ktar, Vear) = softmar(
Limitations of previous work. Existing approaches 1 3 achieve specified-style image gen- eration (or style transfer) through direct feature substitution: Attention (Qref Ktars Vear) Attention (Qtar. Krefi Vrefi). However, this substitution paradigm encounters critical limita- tions when handling structurally divergent reference-tane target pairs, particularly for sketches where sparse representations amplify domain gaps. As visualized in column 2 of Fig. B(a). such methods erroneously focus on local texture replication rather than coherent stroke synthesis, resulting in structural incoherence and artifactual patterns.
Proposed feature injection approach. As evidenced in column 4 of Fig. Bla), simple yet effective feature concatenation strategy alleviates these limitations. The revised attention computation is Vtar
However, this approach still introduces localized chaotic strokes in certain regions. For natural image synthesis, Style
mathematically formulated as Atdention Qtarn
Kior Kref-
))
Vrefi

---

Referenced Styles
0KK AGaIN (StyeAllgred)
Ours twithut Linear 5 Smoothing)
Our twith Linear moothing)
Referenced Styles
Without counter- basad reguiation
With counter. bosad regulation
KWV Swap
sketch of brogo over narrww river
A sketch of squiral holding an scom
A sketch single feother failing through ai'
sketch fruit basket with tru
(31
(b)
Figure (a) Examples of generated results by different K/V injection method. Direct K/V substitution and AdaIN constraints (i.e., StyleAligned [12) introduce visual artifacts (chaotic strokes), whereas our feature concatenation strategy improves line quality. Further incorporating linear blending enhances structural coherence by mitigating content leakage. (b) Counter-based regulation guidance
(Section 3.4) achieves effective artifact suppression with controlled trade-off in stroke fidelity.
12 takes this strategy and attempts enhance output quality through statistical alignment by modulating: Qtar AdaIN AdaIN(QQrr. Qrefi) and Kiar Adal N (Ktor, Krefi). ), where the AdaIN is
x m(x)) a(()) ("
Ada = =vv)
+ p(y).
Unfortunately, different from the scenario of natural images, our experiments (see Fig. 3(a), column 3) demonstrate that such rigid statistical constraints harm sketch generation quality. Instead of this constraint, the key insight underlying M3S is blending deep features from the target content into the reference K/ /V features, strategically trading style consistency for enhanced visual plausibilityy This realized through a linear smoothing operation with the hyperparameter À E [0,1] 1]:
Attention (Qrar:
Ktar Krej.
].
Vear Vrefe [.vrer 1).
Kyef, = AK + (1 A) Krff Vrefi = AVtar + (1 A)V
(3)
Increasing À generally enhances aesthetic quality and text alignment. However, as our base model is trained on natural images, excessively high À risks style degradation and naturalistic outputs over sketches. To synthesize sketches combining styles from both I and Lrefz. we extend Eq.(3) by sim- ply concatenate additional features Krefa AK (1-A) K) and Vrefz >Vaar Whar+& + V) The attention can be written as Attention (Qtar, [Karr Krefs Krefs, (Vtar. Vress 1
3.2 Control the Style Tendency
Inspired by M. we introduce AdaIN modulation for latent noise images to address the color distribu- tion shift problem. This process for single style can be formalized as z! = N xz//) where z/ar is the latent target image in denoising time step t and z refi is obtained by the null-text inversion technology [6]. Next, we discuss the impact the modulation on sketch style. Assuming sketch is represented in bitmap way (0 for strokes, for background), referenced style sketches with dense strokes (e.g., >50% black pixels) exhibit low mean values. AdaIN modulation consequently bi- ases generated sketches toward lower mean value yielding detailed outputs. Conversely, abstract sketches with sparse strokes demonstrate higher μ4, producing more minimalist results. Motivated by this analysis, we introduce a Joint AdaIN module for multi-style tendency control:
tar (z far refi + (1 7) Ado/N (z .Z, refa),
(4)
where 1 [0, is the tendency parameter. Notably, the synthesized results maintain multi-style characteristics even at parameter extremes (n) 0 or 7) 1), as the self-attention calculation incorporates more than single stylistic feature.
5

---

3.3 Style-Content Guidance
While text-to-image diffusion models excel at natural image synthesis, their inherent bias toward photorealistic outputs poses challenges for stylized sketch generation. Conventional classifier- guidance (CFG) [14 combined with feature injection (Section 3.1] and AdaIN modulation (Section 3.2) struggles to maintain stylistic consistency due to fundamental domain discrepancies between
natural images and sketches. To address this, we introduce null-text conditioned style guidance term that establishes dual control pathways:
El = <z, far 0) + wi (Eg (z)"", t. t,tett, Krej, Vres) - cz(z, far , 0))
content guidance divection
(5)
+ Wy (eg (z)"r, ,t,0, Kref Vref)
co(z) tar 0)),
style guidance direction
where cg (-) denotes the noise predicted with feature injection (Section 3.1p. wa and W2 are the content and style guidance scales, respectively. The parameters w/ and require careful calibration in practice achieve an optimal balance between style and content, Excessively high values of w2 may compromise text-image alignment, while insufficiently low values result in uncontrollable stylistic expression in synthesized sketches.
3.4
Counter-based Regulation Guidance
To mitigate potential artifacts in abstract sketch generation with SD v1.5
33 (Fig.
b)), we first V-
apply Tweedie's s formula 20 estimate the denoised latent representation z
Tlee 13 40), which is decoded to image tor- Subsequently, we extract directional gradients through Sobel operators [3] grad, S² 1
lar: grady Sy 1h/r where denotes convolution. Then, we calculate the regular term loss and optimize the latent representations as
Cedge tar OfF
Ledge Igradzl Igradyl,
20
201e
vV
(6)
The updated oo is subsequently applied to calculate х with DDIM step 40 In practice,
empirically set y = 60 as the default and clamp grad and grady within [-0.001, 0. 0.001] to prevent over-amplification of gradient magnitudes that could degrade generation quality.
4
Experiments
Dataset and Metrics We evaluate different methods for single-style referenced generation on six diverse sketch datasets encompassing professional, amateur, and abstract styles: four professional styles from 4skst [37 (Styles 1-4, artist-drawn with referenced images), a web-collected diverse style set (Style 5. 20 sketches from open-source platforms). and 50 abstract freehand sketches from Sketchy (35) (Style 6). For systematic testing, we generate 50 textual prompts via DeepSeek [27] using the template "A sketch of ...", pairing each prompt with randomly selected referenced sketch. To evaluate the performance of M3S and baselines, CLIP-T (32 is used to measure the alignment between generated sketches and prompts. For style consistency, we use the similarity between referenced and target images with the extracted features from DINO [1]. Similar to I the distance of Gram matrices calculated from VGG (38) also considered.
Implementation Details We implement M3S on both Stable Diffusson v v1.5 [33 and SDXL [29]. with all visual results this paper generated by M3S (SD v1.5) unless otherwise specified (more M3S (SDXL)'s 's results are in the Appendix). For M3S (SD v1.5), we configure W1 15, wا = 15 and À 0.1 Styles 1-5, while using W1 = 15, W2 25 and
0.05 for Style Similar to M. feature injection is applied to self- eelfnattention layers in the UNet decoder where feature map resolutions are 32 32 and 64 64. For M3S (SDXL), we set W1 15, W2 = 15 and X = 0.1 for Styles 1-5, adjusting to = 7.5, we = 20 and À = 0.05 for Style 6. The specific injection layers are detailed in the Appendix. The we parameter linearly increases from w2/3 to w2 throughout the denoising process in both implementations. We applied DDIM [40] to sample the target sketches with 100 steps.

---

Asketch of Scetch off butterfly with wiings
sketch of penched fence
sketch of portrait
sketch of fruit basket
sketch of for
Aketch of lighthouse the
sketch of hanging right
Prompts
cup
log
Referenced Stylos
StyloAignoe
Distilationn
CSGO
StyleStudio
Modulation
InstantStyle
Ours (SOKL)
Ours (SD v1.5]
Figure 4: Qualitative comparison of different methods. Most evaluation cases are challenging cross-domain synthesis scenarios. The referenced images in columns 1-4 are from Style 1-4, columns 5-6 are from Style 5, and columns 7-8 are from Style 6.
Baselines We compare with ttate-of-the-art (SOTA) methods spanning diverse technical paradigms: StyleAligned [12] (self-attention feature injection). RB-Modulation (34] (multi-attention aggregation), AttentionDisillation [63], and methods leveraging specialized style adapters InstantStyle (48), CSGO [52], and StyleStudio (2I] These methods are primarily designed for single-reference scenarios. All baselines implemented with their open-source codes.
4.1 Qualitative Analysis
In Fig. we illustrated the qualitative synthesized results in various styles with different methods. Our method effectively captures referenced style sketch attributes. including stroke thickness, pixel density, and luminance levels, to achieve zero-shot text-aligned sketch synthesis. Compared to StyleAligned [12] and AttentionDistillttinn [63], which preserve stylistic fidelity the cost of content leakage (e.g., the fox erroneously positioned on car in the third to last column) and chaotic stroke patterns (visible AttentionDistillation'ss 's first three columns), our M3S framework maintains strict content-semantic consistency while eliminating structural artifacts. These results demonstrate that excessive feature constraints may degrade sketch quality, particularly in artist-style sketches. Although methods like InstantStyle [48] and CSGO (22 generate aesthetically plausible results aligned with textual prompts, they exhibit discernible deviations from referenced styles and constrained style diversity compared to approach. This phenomenon likely stems from these
7

---

Reference
1-=00
0-2025
Reference
A sketch Nessi
sketch of cheongsam
skotch of like Sneakeer
Figure 5: Examples of generated sketches with two referenced style images. Top: Same prompts are used each row, and the prompts are in the Appendix. We set the style tendency r = 0.5 these cases. Bottom: Results of different control the style tendency.
methods' primary focus on natural image style transfer, where their feature injection mechanisms predominantly operate in text space rather than visual space.
The multi-style fusion capability is partially illustrated through specific examples in Fig. and there are some interesting cases: (1) The apple sketch (top-left) fuses distinct stylistic attributes, clear black contours from one reference, and localized texturing patterns from another. (2) The birthday cake synthesis (top-right). When fusing an abstract freehand butterfly sketch and precisely structured car drawing, the result inherits the butterfly's stroke coloration and the car's orderly linework. Conversely, the cake's contours exhibit amatcurish characteristics when substituting the car reference with other sketches. These examples illustrate our framework's 's capacity selectively blend stylistic elements across references while preserving Nomain-appropriaal stroke characteristics. Our method also enables flexible style interpolation through the Joint AdaIN module, shown at the bottom Fig. 5 These interpolated results may be able to stimulate the user's creativity. For instance, increasing progressively enhances the definition of Messi's beard strokes.
4.2 Quantitative Analysis
Table reports the quantitative results of different methods. Under default parameter settings, our M3S (SDXL) achieves the best average CLIP score of 0.3514, demonstrating superior text alignment. While the DINO score and VGG style loss slightly trail AttentionDistillation (63], this reflects our method's balanced style-content trade-off. To validate flexibility,, we provide additional results with style-oriented parameters (denoted as Ours (SDXL") in the table), where reducing content-guidane wi and increasing style-guidance wa yields style metrics comparable AttentionDistillationn while slightly retaining CLIP score advantages. This highlights M3S's ss user-adjustable controllability for subjective style-content balancing. Notably, methods like StyleStudio (2]) produce high-quality images but exhibit lower CLIP scores, likely due to their outputs resembling natural images over

---

Table 1: Sketch-text alignment and style consistency performance comparison across styles. 'Ours (SDXL") denotes that the parameters of our method are set to w1 7.5, w2 20, and X 0.0.
Stylel
Style2
Style3
Method
CIIP--((1)
DINO(1)
VGG(J)
CLIP-T((1)
DINOXT)
VGG()
CIJIP-T())
DINOC)
VGG(.)
StyleAligned
12
0.3130
0.6691
0.0308
0.3095
0,7064
0.0884
0.3013
0.6309
0.0621
Distiaion
0
0.3305
0.7738
0.0930
0.3320
0.7724
0.0320
0.3225
0.7132
0,0305
CSGO 52
0.3336
0.5276
0.0571
0.3257
0.5:109
0.1370
0.3232
0.5151
0.1018
StyleStudio 21
0.3395
0.5164
0,1873
0.3351
0.5601
0.1954
0,3349
0,53337
0.1790
RB Modulation
54
0.3298
0.3624
0.0592
0.3300
0.3429
0.2085
0.3279
0.3453
0.1733
InstantStyle 48
0.3512
0.4934
0.0417
0.3508
0.4929
0.1577
0.3455
0.4394
0.1321
Ours (SDXL)
0.3607
0.651b
0.0165
0,3556
0.6331
0.0671
0.3422
0.6031
0.0834
Ours (SD v1.5)
0.3307
0.6383
0,0200
0.3452
0.6846
0.0616
0.3116
0.6269
0.0571
Ours (SDXL"
0.3-80
0773
0.0122
0.3340
0.1356
0.011
0.3
0.6870
0.037
Style4
Style5
Style6
CIJPP-T(1)
DINO(T)
VGG()
CLIP-T(1)
DINO(T)
VGG()
CIIP-T()))
DINOC)
VGG()
StyleAligned 12
0.3137
0.6407
0.0244
0.3004
0.5428
0.0445
0.2879
0.4445
0.0300
Attentinnissiilation
0.3222
0.7572
0.0061
0.3377
0.6221
0.0173
0.3289
0.7027
0.0190
CSGO [22
0.3321
0.5131
0.0526
0.3298
0.1288
0.0972
0.3211
0,5012
0.0716
StyleStudio MI
0.3402
0.5100
0.1595
0.3377
0.3539
0.1215
0.3338
0.3612
0.1434
RB-Modulation
0.3178
0.3373
0.0165
0.3217
0.3233
0.0972
0.3221
0.2737
0.0780
InstamiSyyl [18
0.3513
0.4494
0,0262
0,3480
0.4408
0.0601
0,3417
0.51.00
Oürs (SDXL)
0,3612
0.6499
0,015
0.3467
0,3332
0.00301
0,3420
0.6922
0.0225
Ours (SD vl.
0.3518
0.6337
0.0136
0.3494
0.5777
0.0272
0.3405
0.7653
0.0170
Ours (SDXL")
0.3506
0.7212
0.0085
0.3383
0.6328
0.0191
Table 2: The average rating of different methods by the human preference assessment.
StyleAligned
[2]
AttentionDistillatiion
63
CSGO [2]
Style
Rating
2.77
4.28
3.85
4.22
RB-Modulation 33
InstantStyle
48
Ours(SD
Ours (SDXL)
Raiing
4.20
5.08
3.44
6.19
sketches. Similarly, lower CLIP scores in Style 3 may stem from dense black patches in references, whereas Style 4 shows the opposite trend. Each sketch takes about 40 seconds (M3S (SD vl.5)) and 70 seconds (M3S (SDXL)) on an A100 40GB GPU.
We conducted human preference assessment via structured questionnaires evaluate text-to- sketch synthesis performance. Each questionnaire contained: 1) Six randomly selected sets of generated sketches 2) Eight anonymized outputs per set from different models under identical prompts and reference styles. 3) Evaluation criteria: Comprehensive assessment across three di- mensions-text alignment, style consistency, and generation quality. Participants ranked results on a scale of 1-8 (8=optimal). From 58 submissions (avg. completion: 4m),s), we excluded 14 invalid responses (<60s completion/missing rankings), retaining 44 validated questionnaires. The results are shown on Table 2 M3S (SDXL) achieved the highest average score (6.19), and M3S (SD v1.5) secured a strong performance (5.44). We evaluate the statistical significance by rank tests, revealing that M3S (SD v1.5) significantly outperforms all baseline methods except InstantStyle (p-value=0.26). When we align the backbone with InstantStyle (i.e., SDXL), our M3S (SDXL) demonstrates superiority over it (p-value=1.00) x 10-5).
The quantitative results of multi-style generation are shown in Table 3 We generate outputs for each prompt by randomly selecting two reference sketches from the Style 5 (S5) dataset. To specifically validate generation performance with significantly distinct reference styles, we conducted an additional experiment set pairing one randomly selected S5 image with one randomly chosen image from the QuickDraw (QD) dataset [II] per prompt. When the references exclusively originate from S5, M3S maintains text alignment comparable to single-style generation. For style consistency, DINO- refl exhibits a positive correlation, while DINO-ref2 shows negative correlation. At boundary conditions (1) = 0 or 1) of multi-style sketch generation, only one style participates in AdaIN modulation for image generation. For the two reference styles, replacing the pair of S5-S5 with QD-S5 reduces style consistency for both implementations (i.e., SD v v1.5 and SDXL), though M3S (SD v1.5) demonstrates superior robustness. Crucially, M3S (SDXL) struggles to effectively utilize QD's abstract features, evidenced by significantly lower scores of DINO-ref1 than DINO-ref2 in
QD-S5 pairs. This limitation stems from SDXL's high-fidelity optimization 29 its user-tested
superiority over SD v v.5 creates inherent incompatibility with low-quality, abstract datasets like QD.

---

Table 3: Multi-style sketch generation performance under different 7y values.
Imp.
stye
Q-
(TIIP-T)")
DINCwelieT)
DINOwef?(7)
(IIIPTT)))
DINCwerliTT
DINOwef?(
(IIIP-T())
DINOwerlec)
DINOweff(
v1.5
0.724
1/38/7
SD.
00-55
03434
0.3630
0.4399
0.3117
0.3948
0.4296
0.5452
0.4:02
0.4057
y-
CL"P
reti(T)
CLIP-TU
(oE2(")
SDNL
v1.5
00-55
0.1250
0.303S
0.5706
Table 4: Results of ablation experiments with different levels of feature injection.
K/V Swap
==0 0
0.05
== 0.1
0.15
X 0.2
0.25
CLIP-T(1) DINO(T)
0.3120
0.3409
0,3443
0,3473
0.3457
0,3468
0.3458
0.3382
0.8109
0.7437
0.6978
0.6459
0.6161
0.5904
0.5719
0.3707
aesthetic
4.5750
4.7952
4.8892
4.9488
4.9577
4.9843
5.0057
5.0493
4.3 Ablation Study
We conduct the ablation study styles 1-4 with SD V1.5 as the backbone. To evaluate the effectiveness of our feature injection approach, Table reports the average scores across these styles. Aesthetic metric [36 is included to reflect the effect more comprehensively. When employing basic K/V Swap, the output maintains excellent style consistency with a DINO score 0.8109, but its low CLIP score reflects poor alignment between generated sketches and textual prompts. By incorporating reference features as auxiliary inputs (i.e., 0), the CLIP metric improves by 9.26%, companied by an acceptable reduction in DINO. However, the aesthetic score of generated images under this con- figuration drops to 4. 4.7952, significantly lower than the reference style average of 5.0549. Increasing mitigates aesthetic degradation and improves text alignment, but fur- ther erodes style fidelity. Beyond critical A, CLIP gains diminish while style loss intensifies, leading to our exper- imentally determined default of 0.1 as an optimal compromise. Fig. [ demonsttssees the impact of content guidance (wi) and style guidance (wa) on generation re- sults. When wi is fixed, increasing w2 reduces the number of intersections in curvilinear textures on the deer's body, driving the output closer to the reference style. Conversely, enhancing w1 under fixed we improves the visual quality of the generated deer. More ablation results are in the Appendix.
Figure 6: The generated results with different content control scale wi and style control scale W2.
5 Conclusions
We propose novel training-free free framework named M3S for zero-shot sketch synthesis blending different styles. The key insight in our method is to take the referenced features as auxiliary information and modulate the latent noise images with the joint AdaIN module. This modulated approach enables the user to control the style tendency. Additionally, the style-content direction guidance provides the flexibility to balance the fidelity and style consistency. A direction for future research and the functionality not included for now is achieving localized style control, where users can explicitly assign specific styles to particular regions rather than relying on the model's automatic style assignment. This capability would further assist and inspire artistic creation.
Acknowledgement
This work was supported by Shanghai Municipal Science and Technology Major Project, China (Grant No. 2021 (02ISHZZZ0002).
10

---

References
[1] Yuval Alaluf, Daniel Garibi, Or Patashnik, Hadar Everbuch-Elow and Daniel Cohen-Or, Cross- image attention for zero-shot appearance transfer. In ACM SIGGRAPH 2024 Conference Papers, pages 1-12, 2024.
[2] Amirsaman Ashtari, Chang Wook Seo, Cholmin Kang, Sihun Cha, and Junyong Noh. Reference based sketch extraction via attention mechanism. ACM Transactions on Graphics (TOG), 41(6):1-16, 2022.
[3] Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free mutual self-attention control for consistent image synthesis and editing. In Proceedings of the IEEE/CVF international conference computer vision, pages 22560-225777 2023.
[4] Yajing Chen, Shikui Tu, Yuqi Yi, and Lei Xu. Sketch-pix2seq: a model to generate sketches of multiple categories. arXiv preprini
2017.
[5] Jiwoo Chung, Sangeek Hyun, and Jae-Pil Heo. Style injection diffusion: A training-free approach for adapting large-scale diffusion models for style transfer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8795-8805, 2024.
[6] Ayan Das, Yongxin Yang, Timothy Hospedales, Tao Xiang, and Yi-Zhe Song. Chirodiff: Modelling chirographic data with diffusion models. In The Eleventh International Conference on Learning Representations, 2023.
[7] Mathias Eitz, James Hays, and Marc Alexa. How do humans sketch objects? ACM Transactions on graphics (TOG), 31(4):1-10, 2012.
[8] Noa Fish, Lilach Perry, Amit Bermano, and Daniel Cohen-Or. Sketchpatch: Sketch stylization via seamless patch-level synthesis. ACM Transactions on Graphics (TOG), 39(6):1144, 2020.
[9] Yarden Frenkel, Yacl Vinker, Ariel Shamir, and Daniel Cohen-Or. Implicit style-content separation using b-lora. European Conference on Computer Vision, pages 181-198. Springer, 2024.
[10] Leon A Gatys, Alexander S Eeker, and Matthias Bethge. Image style transfer using convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2414-2423, 2016.
[11] David Ha and Douglas Eck. A neural representation of sketch drawings. arXiv preprint arXiv: 1704.04477777 2017.
[12] Amir Hertz, Andrey Voynov, Shlomi Fruchter, and Daniel Cohen-Or. Style aligned image generation via shared attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4775-4785 2024.
[13] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:684006851, 2020.
[14] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207. 12598, 2022.
[15] Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. In Proceedings of the IEEE international conference computer vision, pages 1501-1510, 2017.
[16] Inbar Huberman- Huberman-Spiegelggae Spiegelglas, Vladimir Kulikov, and Tomer Michaeli. An edit friendly ddpm noise space: Inversion and manipulations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12469-12478, 2024.
[17] Phillip Isola, Jun-Yan Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1125-1134, 2017. 2017.

---

[18] Ajay Jain, Amber Xie, and Pieter Abbeel. Vectorfusion: Text-to-svg by abstracting pixel-based diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1911-1920, 2023.
[19] Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, pages 694-711. Springer, 2016.
[20] Kwanyoung Kim and Jong Chul Ye. Noise2score: tweedie's ' approach to self-supervised image denoising without clean images. Advances in Neural Information Processing Systems, 34:864-874, 2021.
[21] Mingkun Lei, Xue Song, Beier Zhu, Hao Wang, and Chi Zhang. Stylestudio: Text-driven style transfer with selective control of style elements. arXiv preprint arXiv:2412.08501, 2024.
[22] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2 Bootstrapping language-image pre-training with frozen image encoders and large language models. In International confeerre on machine learning, pages 19740-19742. PMLR, 2023.
[23] Mengtian Li, Zhe Lin, Radomir Mech, Ersin Yumer, and Deva Ramanan. Photo-sketching: Inferring contour drawings from images. In 2019 IEEE Winter Conference Applications of Computer Vision (WACV), pages 1403-1412. IEEE, 2019.
[24] Tengjie Li, Shikui Tu, and Lei Xu. Sketchedit: editing freehand sketches at the stroke-level. In Proceedings of the Thirty-Thire International Joint Conference on Artificial Intelligence, pages 4461-4465 2024.
[25] Tengjie Li, Sicong Zang, Shikui Tu, and Lei Xu. Liser-pix2sen Learning stable sketch representations for sketch healing. Computer Vision and Image Understanding. 240: 103931, 2024.
[26] Yijun Li, Chen Fang, Jimei Yang, Zhaowen Wang, Xin Lu, and Ming-Hsuan Yang. Universal style transfer via feature transforms. Advances neural information processing systems, 30, 2017.
[27] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412. 19437, 2024.
[28] Kushin Mukherjee, Holly Huey, Xuanchen Lu, Yael Vinker, Rio Aguina-Kang, Ariel Shamir, and Judith Fan. Seva: Leveraging sketches to evaluate alignment between human and machine visual abstraction. Advances in Neural Information Processing Systems, 06667138-67155, 2023.
[29] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.019922 2023.
[30] Tianhao Qi, Shancheng Fang, Yanze Wu, Hongtao Xie, Jiawei Liu, Lang Chen, Qian He, and Yongdong Zhang. Deadiff: An efficient stylization diffusion model with disentangled representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern 2024.
Recognition, pages
[31] Yonggang Qi, Guoyao Su, Pinaki Nath Chowdhury, Mingkang Li, and Yi-Zhe Song. Sketch- lattice: Latticed representation for sketch manipulation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 953-961, 2021.
[32] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-888 8763.876.370 PmL.R 2021.
[33] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
12

---

[34] Litu Rout, Yujia Chen, Nataniel Ruiz, Abhishek Kumar, Constantine Caramanis, Sanjay Shakkot- tai, and Wen-Sheng Chu. Rb-modulation: Training-free personalization of diffusion models using stochastic optimal control, 2024.
[35] Patsorn Sangkloy, Nathan Burnell, Cusuh Ham, and James Hays. The sketchy database: learning to retrieve badly drawn bunnies. ACM Transactions on Graphics (TOG), 35 2016.
[36] Christoph Schuhmann. Improved aesthetic predictor, 2022.
[37] Chang Wook Seo, Amirsaman Ashtari, and Junyong Noh. Semi-supervised reference- based sketch extraction using a contrastive learning framework. ACM Transactions on Graphics (TOG), 2023.
[38] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv::1099 1556, 2014.
[39] Irwin Sobel, Gary Feldman, et al. A 3x3 isotropic gradient operator for image processing. a talk at the Stanford Artificial Project in, 1968:271 1-222,, 1968.
[40] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021.
[41] Jifei Song, Kaiyue Pang. Yi-Zhe Song, Tao Xiang, and Timothy M Hospedales. Learning to sketch with shortcut cycle consistency. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 801-810, 2018.
[42] Yang Song. Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.
[43] Guoyao Su, Yonggang Qi, Kaiyue Pang, Jie Yang. and Yi-Zhe Song. Sketchhealer graph-to- sequence network for recreating partial human sketches. In Proceedings of The 31st British Machine Vision Virtual Conference (BMVC 2020), pages 1-14. British Machine Vision Associa- tion, 2020.
[44] Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Improved texture networks: Maximiz- ing quality and diversity in feed-forward stylization and texture synthesis. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages
2017.
[45] Yael Vinker, Yuval Alaluf, Daniel Cohen-Or, and Ariel Shamir. Clipascene: Scene sketching with different types and levels of abstraction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 41 4146-1156, 2023.
[46] Yael Vinker, Ehsan Pajouheshgar, Jessica Y Bo, Roman Christian Bachmann, Amit Haim Bermano, Daniel Cohen-Or, Amir Zamir, and Ariel Shamir. Clipasso: Semantically-aware object sketching. ACM Transactions on Graphics (TOG), 1144):1-11,200 2022.
[47] Yael Vinker, Tamar Rott Shaham, Kristine Zheng, Alex Zhao, Judith Fan, and Anto- nio Torralba. Sketchagent: Language-driven sequential sketch generation. arXiv preprint arXiv:2411. 17673, 2024.
[48] Haofan Wang, Matteo Spinelli, Qixun Wang, Xu Bai, Zekui Qin, and Anthony Chen. In- stantstyle: Free lunch towards style- pryseepre in text-to-image generation. arXiv preprint arXiv:2404.02733, 2024.
[49] Qiang Wang, Haoge Deng, Yonggang Qi, Da Li, and Yi-Zhe Song. Sketchknitter: Vectorized sketch generation with diffusion models. In The Eleventh International Conference Learning Representations, 2023.
[50] Xiaogang Wang and Xiaoou Tang. Face photo-sketch synthesis and recognition. IEEE transac- tions on pattern analysis and machine intelligence, 31(11):1955-191 2008
[51] Xin Wang, Tengjie Li, Sicong Zang, Shikui Tu, and Lei Xu. Self-supervised learning for en- hancing spatial awareness in free-hand sketch. Proceedings of the Thirty-Third International Joint Conference Artificial Intelligence, pages 5117-5125, 2024.
13

---

[52] Peng Xing, Haofan Wang, Yanpeng Sun, Qixun Wang, Xu Bai, Hao Ai, Renyuan Huang, and Zechao Li. Csgo: Content-style composition text-to-image generation. arXiv preprint arXiv:2408. 16766, 2024.
[53] Ximing Xing, Chuang Wang, Haitao Zhou, Jing Zhang, Qian Yu, and Dong Xu. Diffsketcher: Text guided vector sketch synthesis through latent diffusion models. Advances in Neural Information Processing Systems, 11115566118888 2023.
[54] Peng Xu, Timothy M Hospedales, Qiyue Yin, Yi-Zhe Song, Tao Xiang, and Liang Wang. Deep learning for free-hand sketch: A survey. IEEE transactions on pattern analysis and machine intelligence, 4511)2285-312 2022.
[55] Rui Yang, Xiaojun Wu, and Shengfeng He. Mixsa: Training-free reference-basee based sketch extraction via Graphics, 2024.
IEEE Transactions on Visualization and Computer
[56] Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.0672) 2023.
[57] Ran Yi, Yong-Jin Liu, Yu-Kun Lai, and Paul L. Rosin. Apdrawinggan: Generating artistic portrait drawings from face photos with hierarchical gans. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), June 2019.
[58] Sicong Zang, Shikui Tu, and Lei Xu. Controllable stroke-based sketch synthesis from self-organized latent space. Neural Networks, 137:138-150, 2021.
[59] Sicong Zang, Shikui Tu, and Lei Xu. Linking sketch patches by learning synonymous prox- imity for graphic sketch representation. Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 11096-11103, 2023.
[60] Sicong Zang, Shikui Tu, and Lei Xu. Self-organizing latent hierarchy of sketch patterns for controllable sketch synthesis. IEEE Transactions on Neural Networks and Learning Systems, 2023.
[61] Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung- Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint 2000::2003.03005 2022.
[62] Yuxin Zhang, Nisha Huang, Fan Tang, Haibin Huang, Chongyang Ma, Weiming Dong, and Changsheng Xu. Inversion- based style transfer with diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10146-10156, 2023.
[63] Yang Zhou, Xu Gao, Zichong Chen, and Hui Huang. Attention distillation: A unified approach to visual characteristics transfer. arXiv preprint arXiv:2502.20235, 2025.
14

---

Appendix
The appendix organized into several sections, including more analysis and additional details. These topics are as follows:
Section
we provide preliminaries about diffusion model.
In Section
more implementation details of our M3S are provided.
In Section analyze the effectiveness of the regular term for abstract and sparse sketch
generation.
In Section
we present the influence the content guidance wi and the style guidance w2.
In Section
we discuss the effectiveness of linear smoothing for style injection.
Section
we compare other generative model for sketch synthesis.
In Section
we provide more synthesized results of M3S (SDXL).
In Section discuss the limitations of M3s.
In Section
we discuss the potential social impact of our method.
In Section
we show the selected images of Style 5.
In Section
we provide the prompts used for qualitative analysis.
In Section
we present the prompts for Fig. 3|s multi-style generation.
A Preliminaries
A.1 Denoising Diffusion Probabilistic Models (DDPM)
Denoising Diffusion Probabilistic Models (DDPMs) 13 establish generative framework that learns model the data distribution Qdata (2o) through two complementary phases:
The forward process systematically perturbs data samples through progressive noise injection. Given initial data Øg ~ Idata constructs Markov chain of latent variables {ax [xx)T_ by gradually adding Gaussian noise according to predefined schedule (3.)7- E (0, 1):
Forward Diffusion Process.
T [D) = II (=:
This transforms the complex data distribution into an isotropic Gaussian distribution g(æT) P I).
Reverse Denoising Process. The reverse process learns to invert the diffusion trajectory by it- eratively denoising from ær N I). Since the true reverse transition (æ,, depends on the intractable data distribution qdata (zo), DDPMs approximate it through learned conditional
Gaussian:
(a( N (w:-1i t), Ex ())
(8)
Training Objective. Instead of direct mean prediction, DDPMs adopt a noise prediction parameter- ization. For timestep ł uniformly sampled from {1, I), the network Eş predicts the injected noise through:
Csimple = Exoo,,ellE - cØ( Varao + vI- a (),, 1)
where 04 := Br and = -T 0's. The denoised mean is then derived as:
Be
p 1)
(10)
VI
aj
During inference, DDPMs sample from po( (1, (a.) iteratively from 7 The complete derivation and connections to stochastic differential equations are detailed in [4).
15

---

deferenced
Delault attout
Defutt
A skatch dfa pn tne on
shotch af flaaing caln water
Figure 7: Examples of generated sketches with different settings of
layers.
B More Implementation Details
To eliminate the possible localized shadows in the generated sketches, we followed the practice of [55] and performed brightening operation on the images. Specifically, we set pixels with values exceeding 0.7 where the pixel value range is normalized between and 1.
Our M3S (SDXL) selects some self-attention layers from the upsample UNet branch to perform the feature injection. We first enumerate all attention layers in the decoder, where odd- nudbereee indices correspond to self-attention layers. In practice, layers with indices [1,9, 17,27,2 17,25,33,41,49.55 69, are selected for reference style information injection. The examples in Fig. 7 demonstrate the critical role of the final self-attention layer in style control. When feature injection is disabled at layer 71, the generated images overemphasize textual prompts, resulting in excessive color patches that deviate from the reference style. Conversely, injecting features across all self-attention layers achieves exceptional style consistency but compromises text alignment (e.g., excessive folds generated sailboat sails). This indicates our layer selection strategy inherently balances style-content trade-offs. Users can adopt the default configuration or customize layer choices based on visual examples to meet specific needs.
C Analysis of the Regular Term
The regularization term in Eq. (6 primarily serves to suppress shadows and enhance stroke definition. As demonstrated in Fig. increasing hyperparameter progressively diminishes shadow regions while accentuating strokes, albeit with unintended svar-shann that introduces stroke unevenness. Empirical analysis reveals optimal performance when resides within [40, 60), balancing shadow removal and stroke smoothness. Applying this technique to enhance abstract sketches introduces approximately 5 seconds of additional processing time per image.
D Analysis of the Style-Content guidance
Fig. @demonstrates the effects content guidance (wi) and style guidance (w2) through professional and abstract style examples. For the professional style (left): (1) All configurations yield text- compliant sketches. (2) High W1 values (20/25) introduce extrancous pens, likely due to insufficient style guidance during carly denoising (note We's 's linear scheduling in Paragraph Implementation Details). (3) Consistent with Fig. (6 we observe that higher w1 enhances aesthetics (c.g., streamlined deer faces) and increased we improves style adherence (rule-based non-intersecting für strokes). (4) Balancing trade-offs: At wi = 5. elevating W2 causes structural distortion (deer torso deformation at W2 20). This is mitigated by increasing W1, motivating default W1 15, W2 15.
For abstract styles (right): (1) wi 5, we produces over-detailed faces inconsistent with the referenced style sketch. (2) Higher eliminates excessive details but weakens "yoga" characteristics. We therefore set W1 = 15, w2 = 25 for abstract cases, accepting marginal quality degradation to prevent artifacts like striped pants (visible at W1 = 15, wi 15).
16

---

==
=
Y= 40
Y
Y
sketch of monkey enting peach
sketch rabbi eatting a burger
fuoedo holding gun
7.00
Trump playing guttar
Figure 8: The effectiveness of different of eq. (6) Areas demarcated in red signify regions of substantial shading or lines that are less clearly defined.
10
25
20
25
01
Figure 9; The generated results with different content guidance scale w1 and style guidance scale w2. Left: "a sketch deer". Right: "a sketch person doing yoga".
E Analysis of Linear Smoothing for Feature Injection
To further analyze the impact of linear smoothing parameter A, Fig. 10 visualizes generation results under varying À values, while Table 5 extends the quantitative analysis from Table with complete metries. Key observations include: (I) 0: Excessive focus on reference style introduces chaotic strokes that compromise text alignment (e.g., window artifacts in row 2). (2) À E (0, 0.25): Progres- sive artifact reduction improves visual cleanliness (row 3 fox torso refinement) while retaining core style attributes. (3) X = 0.25: Incipient style degradation manifests as faint roof lines contradicting reference patterns (row 2). (4) X Complete style disengagement yields naturalistic images lacking artistic stylization. Table 5 quantitatively corroborates these findings: decreasing DINO
17

---

Styles
0
À 0.05
==
=.155
=
=
x=1
sketch brigee
sketch of small house with windows
sketch o fox standing log
A sketch of vintage camera on tripod
Figure 10: Examples of different linear smoothing parameter À for feature injection.
Table Results of ablation experiments with different levels feature injection. This Table reports the source data of T
Method
Stylel
Style2
Stylo3
Stylo4
CLIP-T(1)
(INO(())
aesthetic
CLIP-T())
DINOKT)
acsthetic
CLIP-I(1)
DINO(T)
acsthetic
CLIP-T(T)
DING(")
acsthetic
swap
0.3310
4.7506
0.3467
4.7962
0.05
0.3474
DUG876
4.8609
0.3849
0.7257
4.8636
0.3360
0.6816
4.5494
0.3487
0.8923
4.822
0.1
0.3507
0.6383
6.9555
0.3652
0.6846
4.9063
0.3416
0.6269
1.9877
0.3518
0.6337
4.5476
= 0.15
0.3496
0.6153
4.9473
0.3463
0.6306
4.9115
0.3582
0.6005
5.0179
0.3-107
0.6000
4.9641
0.2
0.350]
0.5971
5.0359
0.3463
0.6170
4.9260
0.3404
0.5735
4.9915
0.3500
0.5741
1.9856
0.3382
scores and improving aesthetic metrics with increasing A, reflecting the inherent trade-off between style preservation and content alignment.
F
Comparisons with More Methods
While the main text focuses on style-specific generation methods, Fig. provides a comparative anal- ysis with broader sketch synthesis approaches. Implementations of CL [46] and DiffSketcher 53 follow their official codebases, requiring over minutes per sketch generation. We observe that CLIPasso, DiffSketcher, and SketchRNN (II] produce stylistically homogeneous outputs. Notably, CLIPasso's reliance on foreground extraction networks leads to chaotic strokes when segmentation fails (e.g., indistinguishable floral patterns). Although Stable Diffusion v1.5
33 exhibits style diversity, its outputs demonstrate uncontrolled style variation and struggle to produce sketches with white backgrounds and black strokes. Our method effectively addresses these limitations through controlled style injection while maintaining superior visual quality. When using DiffSketcher's outputs as reference styles, M3S successfully emulates this style (Fig TT bottom), demonstrating seamless integration with specialized models.
G Generated Sketches by M3S (SDXL)
Compared SD v1.5. SDXL employs a significantly larger denoising network architecture and enhances the dimensionality of text-conditioning embeddings. In the main text. we provide several
18

---

Content images for CLiPasse DiffS ketzher
CLiPasso
Difis ketcher
Stable Diffusion v1.5
S SeetchRNN
Ours
Referenced styles
Figure 11: Examples with different methods generate cats and flowers. CLIPasso (46) and DiffSketcher (53] require the content image to extract sketches. The prompt 'a sketch of a cat/flower' used for Stable Diffusion V1.5 (33) and our method. In the last column of our method, we use one of the synthesized sketches of DiflSketcher as the referenced style.
results synthesized by M3S (SD v1.5), and illustrate generated sketches by the more powerful M3S (SDXL).
Benefiting from SDXL's superior text-to-image synthesis capabilities, our M3S framework achieves precise text-aligned sketch generation across diverse artistic styles, even for complex com- positional prompts like "A sketch of cyberpunk-style cat with mechanical limbs" " while maintaining striet adherence to reference style characteristics, as shown in Fig. 12 illustrates the results of multi-style generation.
Fig. 13
M3S (SDXL.) effectively integrates styles from dual reference images through its joint AdaIN modula- tion mechanism for style tendency control. The mirror downside is that the M3S (SDXL) is less than
19

---

perfect in the style migration of local details. For instance, the elephant contour in row 2 demonstrates insufficient adoption of the left reference's segmented-line style, where discrete strokes appear less pronounced. This discrepancy likely stems from SDXL's architectural advancements: deeper network layers, augmented attention modules, and heightened model complexity. Notably, these modifications simultaneously enhance text-alignment precision, particularly for prompts requiring fine-grained semantic grounding.
M3S (SDXL) preserves the intrinsic diversity of diffusion models - generating distinct sketches from identical textual prompts and reference styles through stochastic initial noise sampling, as demonstrated in Fig. [4 This stochasticity manifests across multiple dimensions: (1) Form variation. Divergent animal pose. (2) Facial Articulation. Unique combinations of facial features and hairstyles. (3) Compositional novelty. Alternative spatial arrangements of scene elements. Such multi-faccted diversity provides artists with rich creative inspiration to create sketches.
H
Limitation
Fig. 15 Although our M3S demonstrates superior performance in most scenarios, certain limitations persist. As shown in the left panel of Fig- [5] when reference sketches are excessively sparse with content concentrated in small image regions, M3S struggles to generate complete and clear sketches. This stems from two factors: (I) Extreme sparsity hinders effective style feature extraction for guiding generation, and (2) Higher pixel intensity averages in such references disrupt AdaIN modulation, limiting pixel availability for text-aligned sketch synthesis.
propose potential mitigation strategy (Fig 15 right): Enhancing reference images via zooming and replication-baal based padding to increase pixel density. While this alleviates the issue partially, the augmentation process inevitably alters original stylistic attributes (e.g., sparsity distribution and stroke thickness). Consequently, fully resolving this limitation necessitates further investigation into non-destructive reference adaptation methods.
I Social Impact
As illustrated in Fig. and Fig. 12
M3S enables high-quality artist-style sketch generation with an exemplar. It takes only few tens of seconds to produce work that would take human hours. Meanwhile, as shown Fig. and Fig. [3] our novel multi-style sketch generation technology and style preference control can provide users with more creative inspiration.
Selected Style Images of 5
As illustrated in Fig. [6 we curated sketches spanning diverse artistic styles, including portraiture. scalar vector graphics, meticulous brushwork (gongbi), minimalist aesthetics, and detailed rendering styles, among others.
K Prompts for Quantitative Experiments
The textual prompts for quantitative evaluation cover a wide range common real-life scenarios and categories, including animals, landscapes, vehicles, and daily objects etc. Specific prompts are as follows:
1. a sketch of sailboat floating on calm water
2. a sketch of pine tree on a small hill
3. sketch of cat sitting inside teacup
4. sketch of hot air balloon high over mountains
5. a sketch of dragon flying in the sky, full body
6. a sketch bicycle leaning against a brick wall
7. sketch of small house with two windows
8. sketch of fruit basket with fruits
20

---

Referenced Styles
A skketch of bunny
bay holding an umboella
cyberpunk-alyye
mechanical limbs
sitting on the sola
sketch house the mountaii
skeech of church
Figure 12: Sketches generated by the proposed M3S (SDXL). The results in each column are obtained using the same prompts and different referenced styles.
9. sketch of portrait of naruto
10. a sketch of enchanted forest with glowing mushrooms
11. sketch of James Bond in tuxedo and holding a gun
12. sketch of man performing tai chi
13. a sketch of butterfly with plain wings
14. a sketch campfire under starry sky
15. sketch of sydney opera house
16. sketch of bridge over narrow river
21

---

Referenced Style
Referenced Style
=
0.25
1. =.5
0.75
==
sketch a modern
elephant
sketch of Taylor Swift
sketch coffee mug
A sketch a knight holding sword
A sketch a frog
sketch of spidernan
Figure 13: Generated sketches by M3S (SDXL) with multi-styles. The parameter 7 is used for controlling the style tendency.
17. sketch of robot holding flower
18. a sketch of two white bunnies
sketch of lighthouse by the ocean
20. sketch of owl perched
21. sketch of single feather falling through air
22. a sketch of pair of glasses on an open book
23. sketch of steaming coffee cup on saucer
24. sketch of crescent moon with star
22

---

Referenced Styles
Figure 14: Generated sketches of the same styles and the same prompts in each row. We set different
seeds to synthesize various sketches by M3S (SDXL).
25. a sketch of fox standing on log
26. sketch of an empty swing hanging from a tree
27. sketch of vintage camera tripod
28. sketch of seashell smooth sand
29. sketch paper airplane mid-flight
30. a sketch of single rose in a slim vase
31. sketch of mountain peak piercing clouds
32. sketch of deer standing in meadow
23

---

Referenced styles
Generated results
Augmented referenced styles
Generated results
sketch of a winding staircase in tower
sketch of Sydney Opera House
4
Figure 15: Left: Failure cases of M3S. When the referenced sketches are too small or sparse, M3S difficult produce meaningful results. Right: A potential resolution through image augmentation.
Figure 16: The selected images of style 5.
33. a sketch of hanging lantern at night
34. a sketch winding road through desert
35. sketch of penguin on an ice floe
36. sketch of key lying on wooden table
37. sketch of waterfall cascading down rocks
38. sketch of pair of scissors cutting paper

---

39. sketch of street lamp in light fog
40. a sketch of cactus in a clay pot
41. a sketch violin leaning on a chair
42. sketch of squirrel holding an acorn
43. sketch of single leaf floating water
44. sketch of an open umbrella against wind
45. a sketch of stone arch bridge at dawn
46. a sketch of chess piece on a board
47. sketch of crow perched on fence
48. sketch of winding staircase in tower
49. sketch of person doing yoga pose
50. a sketch of winding river through hills
L Text Prompts for Fig. 5
The textual prompts for the top-right of Fig. each row:
1. sketch of an apple
2. a sketch of teddy bear
3. a sketch of iron man
4. sketch of Albert Einstein
Top-left:
1. a sketch of Eiffel Tower
2. a sketch dog wearing hat
3. sketch of birthday cake
4. sketch of coconut tree
25

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have provided the details the proposed module and extensive experiment results support our claim.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We discuss that the balance between style consistency and content alignment is tradeoff problem in Quantitative Analysis. the Conelusions section, we point out the features that our approach currently leaves unimplemented. We also include a Limitation section in the Appendix.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
26

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
We propose novel framework for sketch generation, but not general theory.
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We have provided the implementation details the main text. The code is in the supplementary materials, and more details are in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
27

---

reproducible, they 5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Question:
Answer: [Yes]
Justification: We attach the code in the Supplementary materials. The dataset 4skst used in our paper is public, and we provide the selected data of style in the Appendix.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https for more details.
/nips. wttpcc/pcccccccccccooooooop httpai/gggudeee/deSuuuubbiiiii wereiisssnnoooiiiio
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: In the Section Experiments, we provide the implementation details, and more information is in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification: The commonly used CLIP-T score and DINO score are used measure the image-text alignment and style consistency.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
28

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative еπτог rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: The experiments compute resources of our method are included in the last
sentence of Quantitative Analysis.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https::/neurips. cc http://rww.pp.cpppaaccccccccaaaa
Answer: [Yes]
Justification:
We meet the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: In the qualitative analysis, we explained why our method can assist artist creation. We also include social imapact section in the Appendix.
Guidelines:
The answer NA means that there is no societal impact of the work performed.

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
The models compared in this paper are open-source.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
30

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssit
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moode as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
31

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
32


Reviews:
[{"review_id": "jhJGjMudSl", "reviewer": "Reviewer_58qS", "summary": "The paper presents a training-free framework for style transfer for sketch generation (image of a sketch, rather than a vector), including the ability to interpolate or use multiple reference styles. The framework is applicable for pretrained diffusion models, and the results are shown on Stable Diffusion / SDXL.\n\nThe core of the proposed technique is to, instead of using the K and V from the style source in the attention mechanism during the generation of the target image, to concatenate the style K and V matrices to the target t ones (potentially after linearly blending them with the target ones. For blending multiple styles, the linear combination of the latent images is used during the generation process.\n\nAuthors perform evaluation based on the style source images in Sketchy and 4skst. The textual prompts are generated using DeepSeek. CLIP-T score is used to measure alignment between prompt and generated images, and the distance in the space of DINO/VGG features is used to measure the style alignment with the reference style image.", "questions": "While the general motivation for introducing the content and style guidance losses (eq 5 is clear), the motivation to linearly increase the weight of the style guidance through the generation process is not clear. Why should this be done and how sensitive is the method to the schedule?\n\nFor the SDXL Implementation, the authors mention that style injection is important for the last layers, but injection into all layers compromises text alignment. Is there a principled way for deciding which layers to use, or some motivating ablations around it?", "limitations": "Yes - the paper addresses many potential issues, and acknowledges a limitation when dealing with extremely sparse reference sketches where content is concentrated in a small area.", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Quality & Clarity: Authors clearly motivate their approach and discuss techniques that didn't work (ex. the use of AdaIN from Style aligned paper in lines 139-140). They perform the ablation study on the proposed modifications and do extensive comparison with state-of-the-art, outperforming it in the majority of cases. The writing is clear and there are plenty of examples in the paper. Supplementary material contains the code necessary for reproducibility for both SDXL and Stable Diffusion 1.5, as well as the source style images.\n\nSignificance & Originality: The proposed technique is novel and well-motivated for the application domain (which is somewhat narrow, as some of the assumptions made by the authors rely, for example, on images being grayscale).", "was_revised": true, "final_justification": "The author rebuttal addressed the style guidance losses and style injection layers. I maintain my positive rating (accept).", "rebuttal": "# Response to Reviewer 58qS\n\nWe sincerely thank the Reviewer for your thoughtful feedback and positive assessment of our work. We answer specific questions below.\n\n### **Q1. While the general motivation for introducing the content and style guidance losses (eq 5 is clear), the motivation to linearly increase the weight of the style guidance through the generation process is not clear. Why should this be done and how sensitive is the method to the schedule?**\n\nA1. We thank the reviewer for the valuable feedback. We linearly increase the style guidance weight during denoising to account for diffusion models' temporal dynamics in generation [a]. Early denoising primarily establishes image structure, while later stages focus on texture refinement. Applying strong style guidance $\\omega_2$ prematurely could disrupt structural integrity due to the absence of textual alignment, i.e., $(\\epsilon\\_\\theta^\\times ({z}\\_t^{tar},t,\\emptyset ) -\\epsilon\\_\\theta({z}\\_t^{tar},t,\\emptyset ))$. For example, when style guidance is not properly scheduled, the prompt \"a sketch of a monkey eating a peach\" might activate the generation of connecting the arm directly to the peach. Quantitative results for Style 1-5 generations using M3S (SDXL) are provided in the Table below. Without linear scheduling, CLIP-T and aesthetic scores decrease while the DINO score increases. This occurs because without linear scheduling, the generation process over-prioritizes style consistency. However, higher style consistency doesn't guarantee better visual quality and may cause severe unnatural artifacts in local content. We therefore employ linear scheduling to **balance generation quality, text alignment, and style consistency**. This technique also implemented in [b].\n\n| Method          |       |         |Style1           |           |         |Style2           |           |         |Style3           |           |\n| :-------------- | :---- | :------------ | :-------- | :-------- | :------------ | :-------- | :-------- | :------------ | :-------- | :-------- |\n|                 |       | CLIP-T(↑)     | DINO(↑)   | asesthetic    | CLIP-T(↑)     | DINO(↑)   | asesthetic    | CLIP-T(↑)     | DINO(↑)   | asesthetic    |\n| With Linear     |       | 0.3607    | 0.6545    | 4.9802 | 0.3556    | 0.6531    | 5.0581   |0.3422 | 0.6041    |5.1605|\n| Without Linear  |       | 0.3571        | 0.6813    |4.8622   | 03515       | 0.6655    | 4.97245    |0.3397        |0.6128   |5.0463    |\n\n| Method          |       |         |Style4           |           |         |Style5           |           | \n| :-------------- | :---- | :------------ | :-------- | :-------- | :------------ | :-------- | :-------- |\n|                 |       | CLIP-T(↑)     | DINO(↑)   | asesthetic   | CLIP-T(↑)     | DINO(↑)   | asesthetic   | \n| With Linear     |       | 0.3612    | 0.6493|5.0580 | 0.3467 | 0.5332    | 5.1983  | \n| Without Linear  |       | 0.3578 | 0.6642    |4.9168    | 0.3460   | 0.5682 | 5.0462 |\n\n### **Q2. For the SDXL Implementation, the authors mention that style injection is important for the last layers, but injection into all layers compromises text alignment. Is there a principled way for deciding which layers, or some motivating ablations around it?**\n\nA2. We appreciate the reviewer's constructive comment. Our layer selection strategy prioritizes deeper near-output blocks. It is motivated by the ablation studies in [c], which has demonstrated that diffusion models generate outlines primarily in lower-resolution blocks and textures in higher-resolution blocks, respectively. Injecting features across all layers causes noticeable content leakage. To balance style transfer with generation quality, we reduce the injection frequency into lower-resolution blocks. We compare the results between M3S (SDXL) and full-layer injection under identical settings in the Table below. Full injection significantly reduces CLIP-T and aesthetic scores, while slightly increasing the DINO score due to content leakage – a similar observation to that in Q1.\n\n| Method          |       |         |Style1           |           |         |Style2           |           |         |Style3           |           |\n| :-------------- | :---- | :------------ | :-------- | :-------- | :------------ | :-------- | :-------- | :------------ | :-------- | :-------- |\n|                 |       | CLIP-T(↑)     | DINO(↑)   | asesthetic    | CLIP-T(↑)     | DINO(↑)   | asesthetic    | CLIP-T(↑)     | DINO(↑)   | asesthetic    |\n| M3S (SDXL)     |       | 0.3607    | 0.6545    | 4.9802 | 0.3556    | 0.6531    | 5.0581   |0.3422 | 0.6041    |5.1605|\n|Full-layer  |       | 0.3462  | 0.6697  |4.7033   | 0.3351   | 0.7092  | 4.7849  |0.3243  |  0.6567  |  4.9101    |\n\n| Method          |       |         |Style4           |           |         |Style5           |           | \n| :-------------- | :---- | :------------ | :-------- | :-------- | :------------ | :-------- | :-------- |\n|                 |       | CLIP-T(↑)     | DINO(↑)   | asesthetic   | CLIP-T(↑)     | DINO(↑)   | asesthetic   | \n| M3S (SDXL)    |       | 0.3612    | 0.6493|5.0580 | 0.3467 | 0.5332    | 5.1983  | \n| Full-layer  |       | 0.3433  |0.6257   | 4.7400   | 0.3348 | 0.5381 | 5.0002 | \n\n\n\n### **Q3. The proposed technique is novel and well-motivated for the application domain (which is somewhat narrow, as some of the assumptions made by the authors rely, for example, on images being grayscale).**\n\nA3. We sincerely thank you for recognizing our work's contributions and noting its limitations. In future work, we will validate our multi-style interpolation framework in the complex RGB space and refine the Joint AdaIN module. \n\n[a] Patashnik, Or, et al. \"Localizing object-level shape variations with text-to-image diffusion models.\" ICCV 2023.\n\n[b] Alaluf, Yuval, et al. \"Cross-image attention for zero-shot appearance transfer.\" SIGGRAPH 2024.\n\n[c] Cao, Mingdeng, et al. \"Masactrl: Tuning-free mutual self-attention control for consistent image synthesis and editing.\" ICCV 2023."}, {"review_id": "DNZFrJbQmf", "reviewer": "Reviewer_cQmr", "summary": "This paper presents a novel training - free diffusion model framework for multiple - style references. By integrating reference features as auxiliary information through linear smoothing, it effectively suppresses reference - sketch content leakage. Meanwhile, via a joint AdaIN module for coordinating multi - style generation, it achieves flexible style control without large - scale training, enhancing synthesis quality and expanding diversity.", "questions": "This method has certain bias when processing certain objects. As shown in the clown example in Figure 1, even with $\\eta$ set to 0, the generated clown image doesn't have a simple drawing style, though clowns are suitable for simple drawing. This echoes the fact that the method doesn't achieve optimal results in all cases in Table 2. What's the deep reason for this?\n\nThe paper only shows results of referring to multiple styles, but the used reference sketches are all of high - quality. So when encountering cases with big differences in style and drawing quality, the method's performance is unclear. For example, what's the effect when using amateur sketches (such as those in QuickDraw) and professional sketches as style references?", "limitations": "Referring to the suggestions put forward in the Questions", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "This paper presents a novel training - free diffusion - model - based framework. It can combine textual prompts and reference - style sketches to achieve accurate style guidance. Experiments show its significant advantages in style - alignment accuracy and control flexibility. The paper is well - structured, with clear method descriptions and detailed arguments. However, the need to adjust many hyperparameters restricts its practical - application convenience.", "was_revised": false, "final_justification": "", "rebuttal": "# Response to Reviewer cQmr\n\nWe thank the reviewer for the constructive comments and positive feedback on our paper. Regarding the concerns of the reviewer cQmr, we provide the following responses.\n\n### **Q1. This method has certain bias when processing certain objects. As shown in the clown example in Figure 1, even with $\\eta$ set to 0, the generated clown image doesn't have a simple drawing style, though clowns are suitable for simple drawing. This echoes the fact that the method doesn't achieve optimal results in all cases in Table 2. What's the deep reason for this?**\n\nA1. We thank the reviewer for the valuable feedback. When $η=0$, M3S synthesizes sketches that **​blend both reference styles​ rather than adopting a single style** in this case, while strategically biasing the output toward the aesthetic attributes of Reference Style 1.  We explain this phenomenon through M3S' three core components:\n\n+ **K/V Injection**: The clown example involves multi-style generation where K/V features from both reference images are injected. The diffusion model automatically assigns contour and texture to different objects. At $\\eta=0$, the unicycle exhibits pure freehand sketch style, and the clown maintains freehand-style contours but develops body textures. This occurs because Stable Diffusion's natural image training data contains more detailed clown depictions (colorful costumes, facial makeup) than unicycles. Consequently, the model prioritizes texture-rich styles for clown details.\n\n+ **Joint AdaIN Module**: AdaIN modulates pixel/feature distributions statistically rather than enforcing explicit structural constraints. Thus, at $\\eta=0$, the result shows global distribution aligns with freehand sketch style and local discrepancies persist in structural details.\n\n+ **Style-Content Guidance**: The phenomenon arises from the interplay between reference style characteristics and guidance scaling. When using sparse (cup) and dense (wave) references, the wave’s rich textures enhance content generation due to their structural relevance. For the clown example, however, the default setting of $\\omega_1 = 15, \\omega_2 = 15$ amplifies texture density from the non-freehand reference style. This occurs because higher $\\omega_2$ prioritizes style alignment, favoring denser patterns. When we reduce $\\omega_2$ to 7.5, the textures for the clown are more line-dominant (closer to freehand). Since OpenReview does not allow to upload images during the discussion period, we recommend that the reviewer verify the results via the code and dataset in the Supplementary materials.\n\nIn essence, this phenomenon reflects the model's automatic selection of multi-style features for category-specific synthesis. Since we inject features from multiple references, the generated image inherently blends diverse styles even at $\\eta=0$. **To achieve pure freehand sketch style, we recommend using sparse references (e.g., the cup) as the sole style source.**\n\n### **Q2. The paper only shows results of referring to multiple styles, but the used reference sketches are all of high - quality. So when encountering cases with big differences in style and drawing quality, the method's performance is unclear. For example, what's the effect when using amateur sketches (such as those in QuickDraw) and professional sketches as style references?**\n\nA2. We appreciate the reviewer's constructive comments. To verify generation quality when using **two visually distinct reference styles**, we experimented with the following design. We first constructed a style dataset by randomly selecting 50 images from the **low quality dataset QuickDraw (QD)**, then generated 50 multi-style images using prompts identical to our main quantitative experiments. Specifically, we select one random reference from QD and one from **high quality dataset Style 5 (S5)** per generation. To contrast results against cases where both references exhibit high quality, we generated additional outputs using exclusively S5 references. Quantitative results are provided in the table below. \n\nThe results indicate that QD-S5 and S5-S5 achieve comparable CLIP-T scores with only marginal degradation, demonstrating our method's robust text alignment even with significantly divergent reference styles. For SD v1.5, DINO metrics show only minor deterioration when using QD references. However, QD-S5 SDXL exhibits substantially reduced DINO-ref1, indicating the model's stronger preference for high-quality S5-aligned outputs over QD. This occurs because SDXL is explicitly optimized for high-fidelity generation and significantly outperforms SD v1.5 in user evaluations [a], making it more susceptible to quality discrepancies in reference images. Consequently, M3S (SD v1.5) is preferable for multi-style generation involving references with major quality differences.\n\n| M3S implementations|Reference style |         |$\\eta=0$  |           |         |$\\eta=0.25$        |           |         |$\\eta=0.5$         |           |\n| :-------------- | :---- | :------------ | :-------- | :-------- | :------------ | :-------- | :-------- | :------------ | :-------- | :-------- |\n|                 |       | CLIP-T(↑)     | DINO-ref1   | DINO-ref2 | CLIP-T | DINO-ref1| DINO-ref2 | CLIP-T | DINO-ref1|DINO-ref2|\n| SDXL | S5-S5 |0.3442   | 0.3936    | 0.4944   | 0.3514    |0.4180   | 0.4821   |0.3495   |0.4408    |0.4556  |\n|SDXL| QD-S5|  0.3426   | 0.3051 | 0.4724   |  0.3455 | 0.3266    |0.4622     |0.3457       |0.3330   |0.4397    |\n|SD v1.5|  S5-S5 | 0.3465    | 0.3850    | 0.4776 | 0.3453    | 0.4215    | 0.4597   |0.3499  |0.4469    |0.4509|\n|SD v1.5|QD-S5| 0.3434        | 0.3630   |0.4339   | 0.3417       |0.3948    |0.4236    |0.3452        |0.4102   |0.4057    |\n\n| M3S implementations|Reference style| |$\\eta=0.75$          |           |         |$\\eta=1$          |           | \n| :-------------- | :---- | :------------ | :-------- | :-------- | :------------ | :-------- | :-------- |\n|                 |       | CLIP-T(↑)     | DINO-ref1 | DINO-ref2| CLIP-T(↑)  | DINO-ref1 |DINO-ref2| \n| SDXL | S5-S5 |0.3499    | 0.4578|0.4221   | 0.3470   | 0.4693    | 0.3975  | \n| SDXL| QD-S5|    0.3447       | 0.3409    |0.4209 |0.3396       | 0.3617    |0.3916 |\n| SD v1.5|  S5-S5 | 0.3478    | 0.4528|0.4257 | 0.3528 | 0.4626    |0.3825  | \n|SD v1.5|QD-S5| 0.3440 | 0.4250    |0.3938   | 0.3468   |0.4381 |0.3766 |\n\n\n### **Q3. However, the need to adjust many hyperparameters restricts its practical application convenience.**\n\nA3. Thanks for your thoughtful comments. Since human evaluation of text-conditioned sketch generation typically requires multi-factor considerations, style-specific generation frameworks commonly offer multiple parameter values for user customization [b,c]. Similar to the literature, our method also has tunable hyperparameters. We provide default parameter values for both professional and abstract sketches to satisfy user requirements, as evidenced by the user study (**See Response to W249, Q1**).\n\n[a] Podell, Dustin, et al. \"Sdxl: Improving latent diffusion models for high-resolution image synthesis.\" arXiv 2023.\n\n[b] Lei, Mingkun, et al. \"StyleStudio: Text-Driven Style Transfer with Selective Control of Style Elements.\" CVPR 2025.\n\n[c] Zhou, Yang, et al. \"Attention distillation: A unified approach to visual characteristics transfer.\" CVPR 2025."}, {"review_id": "PhmA1V2sqq", "reviewer": "Reviewer_W249", "summary": "This paper proposes a method called M3S for generating stylized sketches from a text prompt, guided by one or two reference sketches. Instead of training a new model, the authors use a pre-trained Stable Diffusion model and inject style features by modifying the key (K) and value (V) components in the attention layers of the pre-trained model. They also leverage an AdaIN-based control mechanism that enables users to interpolate between different sketch styles. They propose a dual guidance formulation that separates text (content) and style impact, allowing for more fine-grained control over the generated sketches. Quantitative results suggest partial improvement over existing methods. This model also supports multi-style blending, which can be the unique feature of this model.", "questions": "To strengthen the paper, I recommend the following improvements:\n\n**1. [Conducting a user study]:** Due to the subjectivity of sketch aesthetics, conducting a user study could help to verify this method superiority to existing baselines.\n\n**2. [Handling diverse reference styles]:** I would like to see how this model handles interpolating between styles, especially when two styles are really different from each other.\n\n**3. [Eq. 5 might be better represented]:** I think authors might improve the representation of Eq. 5. After reading the text, I could figure it out but it should somehow be more clear in the equation (somehow it should show where blended K/V features are used). \n\n**4. [Failure cases]:** It will be great if authors mention which kind of styles result in failure of this model. For example, can this model imitate the style of pointillism sketches (sketches drawn by many points)?\n\n**5. [Few missing references]:** Some reference-based sketch extraction studies from the computer graphic domain might be missing (see below). This is not a main concern; and I trust the authors to add if they found new ones; if not, it was a very minor comment that they can skip it. Here I added examples of missing studies that introduced style transfer concept into the sketch domain: [1] SketchPatch: Sketch stylization via seamless patch-level synthesis. TOG 2020, [2] Reference Based Sketch Extraction via Attention Mechanism, TOG 2022.\n\nI am on the positive side for this paper. If authors clearly demonstrate that their method is outperforming existing baselines or addressing some of the main concerns (like human evaluation), I am willing to increase the score after reading the author comments.", "limitations": "It can be improved by showing failure cases, as it was explained in the review.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Thanks for the authors effort to prepare the manuscript to advance the reference-based sketch extraction domain. Here, I listed the strengths and room for improvement:\n\n**- Strengths:**\n\n**1. [Simple yet effective model]:** I like the simplicity of the proposed method. It does not require re-training and it is built upon the existing knowledge in feature fusion and blending. It uses existing methods (e.g., K/V blending, AdaIn, and dual guidance) to merge style and content features in a smart way. \n\n**2. [Simple control over generated sketch for a human user]:** From a user point of view, the method seems to give lots of freedom in the generated sketch. Users can easily define two reference styles and control the generated sketch to be similar to one of those styles or to even have in between styles. \n\n---------------\n\n**- Room for improvement:**\n\n**1. [Lack of a user study]:** Sketches are sparse and sometimes quantitative metrics might not be the best representative of human opinion and evaluation. I believe a user study might be needed to show how much this method outperforms existing baselines (especially considering that in some quantitative metrics, this method seems to not be the best performing one).\n\n**2. [Limited technical novelty]:** The proposed method builds on many existing ideas in the literature, which could be seen as a limited technical novelty. However, if the model clearly outperforms previous baselines, I would consider this a minor issue. For example, AdaIN blending has long been used for arbitrary style transfer. K/V injection (substitution) in diffusion attention layers has been done before and here the feature fusion is improved to avoid artifacts.\n\n**3. [Quantitative evaluation of multi-style interpolation]:** While style blending is demonstrated visually, there are no metrics provided to quantify the quality of the style interpolations; such as the smoothness of transitions or the similarity of interpolated sketches to either reference (ref1 or ref2). It is also important to evaluate the consistency of interpolation quality across different pairs of reference styles, especially when the styles differ significantly from each other.", "was_revised": true, "final_justification": "Thanks for authors for addressing my concerns. I raised my initial positive score.", "rebuttal": "# Response to Reviewer W249\n\nWe thank the reviewer for the constructive comments and positive feedback on our paper. Regarding the concerns of the reviewer W249, we provide the following responses.\n\n### **Q1. Conducting a user study.**\n\nA1. We appreciate the reviewer's constructive comments. We have conducted a **user study** employing a questionnaire-based methodology to assess human preference. Each questionnaire included six randomly selected sets of generated results. Every set contained eight outputs produced by different models using identical prompts and reference styles. Participants were instructed to rank all eight anonymized results per set based on **text alignment, style consistency, and generation quality**. The user rankings for each of the eight models are converted to scores from 1 to 8 (8=best) correspondingly. We collected 58 questionnaires with an average completion time of 4 minutes and 16 seconds. After excluding 14 invalid submissions with completion time under 60 seconds and missing rankings, 44 valid responses remained. Average scores per question for each method are tabulated below. \n\nIn the user study, M3S (SDXL) achieved an average score of **6.19** while M3S (SD v1.5) scored **5.44**. We evaluate the statistical significance by **rank tests**, revealing that M3S (SD v1.5) significantly outperforms all baseline methods except InstantStyle (p-value=0.26). When we align the backbone with InstantStyle (i.e., SDXL), our M3S (SDXL) demonstrates superiority over it **(p-value=1.06e-5)**. This demonstrates M3S's superior balance across style alignment, text fidelity, and generation quality. Despite InstantStyle's relatively weaker style consistency, its high aesthetic quality yielded a competitive score of 5.08. Conversely, StyleAligned's advantages in style consistency were undermined by content leakage issues. AttentionDistillation achieved marginal score improvements despite inferior generation quality compared to StyleStudio and RB-Modulation, primarily due to its style consistency. The results affirm M3S's alignment with human preferences.\n\n|| StyleAligned | AttentionDistillation | CSGO | StyleStudio | RB-Modulation | InstantStyle | Ours (SDXL) | Ours (SD v1.5) |\n| :------- | :----------- | :------------------- | :--- | :--------- | :------------ | :----------- | :---------- | :------------- |\n| Rating| 2.77| 4.28 | 3.83 | 4.22 | 4.20 | 5.08 | **6.19**| 5.44 |\n\n### **Q2. Handling diverse reference styles and quantitative evaluation of multi-style interpolation.**\n\nA2. Thanks for your valuable feedback. Following your suggestion, we have added supplementary experiments for multi-style generation. Using the same 50 prompts as the manuscript, we generated outputs for each prompt by randomly selecting two reference sketches from the Style 5 (S5) dataset. To specifically validate generation performance with significantly distinct reference styles, we conducted an additional experiment set pairing one randomly selected S5 image with one randomly chosen image from the QuickDraw (QD) dataset per prompt. **It is noted that the styles of the QD sketches are very different from those in S5.** The quantitative results are provided in the table below.\n\n| M3S implementations|Reference style| |$\\eta=0$ |    | |$\\eta=0.25$  | | |$\\eta=0.5$ ||\n| :-------------- | :---- | :------------ | :-------- | :-------- | :------------ | :-------- | :-------- | :------------ | :-------- | :-------- |\n| | | CLIP-T(↑)     | DINO-ref1   | DINO-ref2 | CLIP-T | DINO-ref1| DINO-ref2 | CLIP-T | DINO-ref1|DINO-ref2|\n|  SDXL | S5-S5 |0.3442| 0.3936 | 0.4944| 0.3514|0.4180 | 0.4821 |0.3495 |0.4408|0.4556  |\n|SD v1.5 |S5-S5| 0.3465| 0.3850 | 0.4776 | 0.3453|0.4215| 0.4597|0.3499|0.4469 |0.4509|\n| SDXL | QD-S5 |0.3426|0.3051|0.4724|  0.3455 | 0.3266|0.4622|0.3457 |0.3330|0.4397|\n|SD v1.5 |QD-S5|0.3434| 0.3630|0.4339| 0.3417|0.3948 |0.4236|0.3452 |0.4102|0.4057|\n\n|M3S implementations|Reference style||$\\eta=0.75$|||$\\eta=1$||\n|:-------------- |:---- | :------------ | :-------- | :-------- | :------------ | :-------- | :-------- |\n|||CLIP-T(↑)|DINO-ref1 |DINO-ref2|CLIP-T(↑)|DINO-ref1|DINO-ref2| \n|SDXL|S5-S5|0.3499|0.4578|0.4221|0.3470|0.4693|0.3975| \n|SD v1.5|S5-S5|0.3478| 0.4528|0.4257|0.3528|0.4626|0.3825| \n|SDXL|QD-S5|0.3447|0.3409|0.4209|0.3396|0.3617|0.3916|\n|SD v1.5|QD-S5|0.3440|0.4250|0.3938|0.3468|0.4381|0.3766|\n\nWhen the references exclusively originate from S5, M3S maintains text alignment comparable to single-style generation (CLIP-T: M3S (SDXL) 0.3467, M3S (SD v1.5) 0.3494). For style consistency, DINO-ref1 exhibits a positive correlation with $\\eta$ (i.e., increasing $\\eta$ enhances alignment with reference 1), while DINO-ref2 shows a negative correlation. This monotonic relationship aligns with qualitative results in Figs. 1 and 5. At boundary conditions ($\\eta$=0 or 1) of multi-style sketch generation, only one style participates in AdaIN modulation for image generation. Compared to single-style synthesis using that style alone, the DINO metric remains relatively ​lower​ due to persistent multi-style feature injection (**see Response to Reviewer cQmr Q1** for more analysis). \n\nFor the two reference styles, replacing the pair of S5-S5 with QD-S5 reduces style consistency for both implementations (i.e., SD v1.5 and SDXL), though M3S (SD v1.5) demonstrates superior robustness. Crucially, M3S (SDXL) struggles to effectively utilize QD's abstract features, evidenced by significantly lower scores of DINO-ref1 than DINO-ref2 in QD-S5 pairs. This limitation stems from SDXL's high-fidelity optimization [a] - its user-tested superiority over SD v1.5 creates inherent incompatibility with low-quality, abstract datasets like QD. Due to OpenReview's restrictions on image uploads during the discussion phase, we cannot directly provide corresponding visual results. However, the overall generation quality is visually analogous to the butterfly reference outputs in Fig. 5 of the main paper.\n\n### **Q3. Eq. 5 might be better represented.**\n\nA3. We appreciate your suggestion. We will revise Eq. (5) to explicitly represent the $K/V$ injection mechanism as follows. Our formulation follows the notation established in [b]. We will also add explanatory text regarding the equation and cite [b] to enhance readability.\n\n$\\tilde{\\epsilon}\\_t=\\epsilon\\_{\\theta}(\\textbf{z}\\_t^{tar},t,\\emptyset)+\\omega\\_1\\underbrace{(\\epsilon\\_\\theta^\\times(\\textbf{z}\\_t^{tar},t,text,K^{ref},V^{ref}) -\\epsilon\\_\\theta(\\textbf{z}\\_t^{tar},t,\\emptyset ))}\\_{content\\ guidance\\ direction} + \\omega\\_2 \\underbrace{(\\epsilon\\_\\theta^\\times (\\textbf{z}\\_t^{tar},t,\\emptyset,K^{ref},V^{ref}) -\\epsilon\\_\\theta(\\textbf{z}\\_t^{tar},t,\\emptyset ))}\\_{style\\ guidance \\ direction}$\n\n\n### **Q4. Failure cases.**\n\nA4. We apologize for placing the failure cases in an insufficiently prominent location (specifically, **Appendix H and Fig. 15** of manuscript). In the upcoming revision, we will incorporate these directly into the main body. Our method's primary failure mode occurs when reference images are excessively sparse with content concentrated in small regions, potentially yielding near-blank outputs. To address this, we introduce a solution in Appendix H as follows, i.e., enhancing reference images via zooming and replication-based padding to increase pixel density. \n\nFollowing your suggestion, we tested our method using Google-collected pointillism sketches (depicting flowers, balls, tigers, etc.). Our method succeeded in most examples. However, we observed **one failure case** where **SD's VAE failed to accurately reconstruct the reference image**, indicating inaccurate latent space representations. This example will be incorporated into the failure cases section in the revised manuscript.\n\n### **Q5. Few missing references.**\n\nA5. We appreciate your feedback on the missing literature. In the revised manuscript, we will incorporate the referenced computer graphics papers relevant to our work and conduct an additional literature review to provide readers with a more comprehensive domain context.\n\n### **Q6. Limited technical novelty.**\n\nA6. Thanks for your constructive feedback. While building upon prior concepts, M3S extends existing techniques in novel directions to address unique challenges in **​multi-style sketch synthesis**: \n\n1. **Multi-reference feature fusion**\n   Prior K/V injection methods (e.g., [b,c,d]) focused on single-style transfer. M3S introduces: **Concurrent injection​ of features** from multiple references (Section 3.1) and blending them through linear smoothing.\n   \n2. **Adaptive noise modulation**\n   Traditional AdaIN [e] achieves style transfer by modulating image features, but its outputs remain constrained by the content image. Recent diffusion-based methods [b] primarily focus on artifact removal without exploring AdaIN's style modulation capabilities. In contrast, **M3S demonstrates that multi-style fusion can be achieved by applying AdaIN to denoising trajectories in noise space, free from encoded-space limitations**. Furthermore, M3S leverages diffusion models' generative capacity to automatically **produce creative content** under varying $\\eta$ conditions, such as the evolving beard details in Messi's portrait (Fig. 5).\n3. **Performance Validation**\n   M3S demonstrates consistent advantages in human evaluations, which achieves highest user preference score (6.19​ for SDXL). \n\nIn summary, our core innovation centers on both the novel task of multi-style sketch synthesis and its corresponding framework design. Building on this framework, we will further explore the feasibility of multi-style fusion generation in RGB space.\n\n\n[a] Podell, Dustin, et al. \"SDXL ....\" arXiv 2023.\n\n[b] Alaluf, Yuval, et al. \"Cross-image attention ...\" SIGGRAPH 2024.\n\n[c] Hertz, Amir, et al. \"Style aligned ...\" CVPR 2024.\n\n[d] Cao, Mingdeng, et al. \"Masactrl ...\" ICCV 2023.\n\n[e] Huang, Xun, et al. \"Arbitrary style transfer ...\" ICCV 2017."}, {"review_id": "Mbb9ugGTW9", "reviewer": "Reviewer_kzEp", "summary": "The paper proposes a text-to-sketch generation method. The method use reference features as auxiliary information with linear smoothing and use a style-content guidance mechanism. The framework can be extended to handle multiple styles (interpolating between different styles). Experimental results show that the proposed method can provide better style-content disentanglement.", "questions": "- As mentioned in \"Strengths And Weaknesses\", the generation speed might be an issue. It would be good to list the break down of time usage for different modules so that the readers can better understand the bottleneck of the proposed method.\n\n- The paper compares with many general baseline works that are designed for general styles. It would be good if the author can clearly mark which methods are designed for general styles, while which methods are designed for sketches only. This will help readers understand the scope of the baseline approaches.\n\n- It is usually subjective to evaluate the quality of a sketch. It would be more convincing if user study results are included in the paper.", "limitations": "The authors have discussed the limitations.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n+ The experimental results shows that the proposed text-to-sketch method outperforms existing baseline approaches, both quantitatively and qualitatively.\n+ The explanation of the proposed method is clear and the paper is easy to follow.\n\nWeaknesses:\n- The main weakness is that text-to-sketch generation might be a sub-domain of text-to-image generation. It is questionable whether it is necessary to design a sophisticated method for the text-to-sketch task. The author mentioned in the paper that existing text-to-image models are not good at sketch generation, e.g., they are designed for color and natural images. However, this problem might just come from training data. If we replace training data, or finetune the existing model with additional sketch images, or train a sketch lora, this problem might have been solved. Different trigger words can be added to training captions to incorporate different sketch styles. \n\n- The proposed method uses null-text-inversion. However, this method usually requires optimization and make the method slow. The paper mentions \"Each sketch takes about 40 seconds (M3S (SD v1.5)) and 70 seconds (M3S (SDXL)) on an A100 40GB GPU\", but there is not comparison with baseline approaches about generation time. The proposed method might be significantly slower than existing approaches, limiting the applications of this approach.", "was_revised": true, "final_justification": "The rebuttal answered my questions about speed, motivation, and generalizability. Thus I am raising my rating", "rebuttal": "# Response to Reviewer 58qS\nWe thank the reviewer for the constructive comments and feedback on our paper. Regarding the concerns of the reviewer 58qS, we provide the following responses.\n\n### **Q1. As mentioned in \"Strengths And Weaknesses\", the generation speed might be an issue. It would be good to list the break down of time usage for different modules so that the readers can better understand the bottleneck of the proposed method.**\n\nA1. Thanks for your valuable feedback. We employ the null-text inversion technique via **DDPM-Inversion** [a], which constructs the latent noise sequence through a forward process, computes and retains the true noise values, and directly utilizes these values during sampling to estimate the target noise—enabling perfect reconstruction. In other words, it can be considered an optimized version of DDIM-Inversion [b]. Crucially, **this is not an optimization-based approach**, thus avoiding the multi-minute optimization time required by methods like [c]. For single-style generation tasks, our method's average inference time per image is reported below:\n\n + **M3S (SD v1.5)​: 41 seconds in total, where 6 seconds for DDPM-Inversion trajectory collection and 35 seconds for denoising synthesis.**\n\n+ **M3S (SDXL)​: 63 seconds in total, where 10 seconds for DDPM-Inversion trajectory collection and 53 seconds for denoising synthesis.** \n\nThe table below records the inference time per sketch generation across different models. Our M3S (SD v1.5) demonstrates slower generation than InstantStyle but comparable speed to CSGO, StyleStudio, and RB-Modulation. While M3S (SDXL) requires additional processing time, it outperforms StyleAligned and AttentionDistillation in efficiency. Notably, InstantStyle, CSGO, RB-Modulation, and StyleStudio rely on **external pre-trained models** beyond their diffusion backbones, incurring non-inference overheads (e.g., RB-Modulation's SAM [d] loading takes minutes). Although InstantStyle offers speed advantages, its dependency on IP-Adapter [e] **necessitates model-specific pre-training**. In contrast, M3S, StyleAligned, and AttentionDistillation—despite longer inference—achieve superior style consistency with **training-free, plug-and-play flexibility**.\n\n|| StyleAligned | AttentionDistillation | CSGO | StyleStudio | RB-Modulation | InstantStyle | Ours (SDXL) | Ours (SD v1.5) |\n| :------- | :----------- | :------------------- | :--- | :--------- | :------------ | :----------- | :---------- | :------------- |\n| Time/Second|76| 98 | 38 |37 | 51 | 13 | 63 | 41 |\n\n### **Q2. The paper compares with many general baseline works that are designed for general styles. It would be good if the author can clearly mark which methods are designed for general styles, while which methods are designed for sketches only. This will help readers understand the scope of the baseline approaches.**\n\nA2. Thanks for your suggestions. The baseline methods, including InstantStyle, CSGO, StyleStudio, RB-Modulation, AttentionDistillation, and StyleAligned are designed for **general scenarios**. Other methods, like SketchRNN, CLIPasso, \nand DiffSketcher are **specifically built for sketch generation**. We will clarify this categorization more explicitly in the revised version. Extensive comparisons with general-purpose methods are conducted because the scarcity of same-style sketch data, e.g., the 4skst [f] dataset contains only 25 sketches per style across 4 styles, makes it impractical to train dedicated style-specific text-to-sketch models. Addressing this data scarcity to achieve zero-shot style-specific generation is a core challenge our work tackles.\n\n### **Q3. It is usually subjective to evaluate the quality of a sketch. It would be more convincing if user study results are included in the paper.**\n\nA3. We appreciate your constructive comments. We conducted a **​human preference assessment**​ via structured questionnaires to evaluate text-to-sketch synthesis performance. Each questionnaire contained:\n1. Six randomly selected sets of generated sketches.\n2. Eight anonymized outputs per set from different models under identical prompts and reference styles.\n3. Evaluation criteria: Comprehensive assessment across three dimensions—text alignment, style consistency, and generation quality.\n\nParticipants ranked results on a scale of 1-8 (8=optimal). From 58 submissions (avg. completion: 4m16s), we excluded 14 invalid responses (<60s completion/missing rankings), retaining ​44 validated questionnaires. The results are shown on the table below.\n\nQuantitative analysis of user preferences revealed:\n1. **​M3S Dominance**:\n   + M3S (SDXL) achieved the highest average score (**6.19**) and M3S (SD v1.5) secured strong performance (**5.44**).\n   + Both variants significantly outperformed baseline methods, demonstrating excellent balance across evaluation dimensions.\n   + We evaluate the statistical significance by **rank tests**, revealing that M3S (SD v1.5) significantly outperforms all baseline methods except InstantStyle (p-value=0.26). When we align the backbone with InstantStyle (i.e., SDXL), our M3S (SDXL) demonstrates superiority over it **(p-value=1.06e-5)**.\n2. **Baseline Comparisons**:\n   + ​InstantStyle​ attained a competitive score (5.08) due to high aesthetic quality despite style consistency limitations.\n   + ​StyleAligned​'s style advantages were compromised by content leakage issues.\n   + AttentionDistillation​ achieved marginal improvements over StyleStudio/RB-Modulation through style-focused optimization, notwithstanding quality deficiencies.\n\nIn conclusion, the study affirms M3S's superior alignment with human evaluator preferences. We will include the results and analysis in the revision version.\n\n\n||StyleAligned|AttentionDistillation|CSGO|StyleStudio| RB-Modulation|InstantStyle|Ours (SDXL)|Ours (SD v1.5)|\n| :------- | :----------- | :------------------- | :--- | :--------- | :------------ | :----------- | :---------- | :------------- |\n|Rating| 2.77|4.28|3.83|4.22|4.20|5.08|**6.19**|5.44|\n\n### **Q4. The main weakness is that text-to-sketch generation might be a sub-domain of text-to-image generation. It is questionable whether it is necessary to design a sophisticated method for the text-to-sketch task. The author mentioned in the paper that existing text-to-image models are not good at sketch generation, e.g., they are designed for color and natural images. However, this problem might just come from training data. If we replace training data, or finetune the existing model with additional sketch images, or train a sketch lora, this problem might have been solved. Different trigger words can be added to training captions to incorporate different sketch styles.**\n\nA4. Thank you for your suggestion for style-specific sketch generation solutions. For specific-style sketch generation, training-based methods face three major challenges in practical applications:\n\n1. **Data Scarcity and Annotation Difficulties**\n   Professionally styled sketch datasets are **scarce and typically small-scale** (e.g., 25 sketches per style in 4skst [f], 140 in APDrawing [g]). Effective fine-tuning requires dozens of samples per style, necessitating thousands of sketches for diverse styles like our Style 5 dataset's 20 distinct categories. Although web-sourced sketches are plentiful, style annotation poses significant hurdles: multimodal models (e.g., BLIP-2 [h]) trained on real image-language pairs will struggle with sketch style categorization. **Manual annotation is costly and inconsistent** due to sketches' abstract nature—style descriptors vary across annotators' age, cultural background, and regional context. Such inaccuracies impede models from learning precise token representations for style triggering.\n\n2. **​Limited Flexibility in Trained Approaches**\n   Training-based methods require model-specific adaptations, whereas our training-free technique transfers seamlessly across architectures. Furthermore, generating novel styles absent from training data necessitates re-training, while M3S operates zero-shot without this constraint. Critically, our framework inherently supports multi-style fusion, whereas trained methods lack this capability without extensive additional research.\n\n3. **​The \"An Image is Worth One Word\" Principle [i]**\n   Given the linguistic ambiguity of sketch styles, users must memorize dozens to hundreds of trigger tokens to match target styles accurately. M3S eliminates this burden by using a single reference image for style guidance, significantly enhancing usability.\n\nOverall, our approach circumvents the data acquisition constraints and flexibility limitations inherent in training-based methods while enabling novel multi-style fused sketch generation. We appreciate your valuable suggestion regarding training-based alternatives and will investigate these directions in future work.\n\n[a] Huberman-Spiegelglas, et al. \"An edit friendly ddpm noise space: Inversion and manipulations.\" CVPR 2024.\n\n[b] Song, Jiaming, et al. \"Denoising diffusion implicit models.\" arXiv preprint arXiv 2020.\n\n[c] Mokady, Ron, et al. \"Null-text inversion for editing real images using guided diffusion models.\" CVPR 2023.\n\n[d] Kirillov, Alexander, et al. \"Segment anything.\" ICCV 2023.\n\n[e] Ye, Hu, et al. \"Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models.\" arXiv 2023.\n\n[f] Seo, Chang Wook, Amirsaman Ashtari, and Junyong Noh. \"Semi-supervised reference-based sketch extraction using a contrastive learning framework.\" TOG 2023: 1-12.\n\n[g] Yi, Ran, et al. \"Apdrawinggan: Generating artistic portrait drawings from face photos with hierarchical gans.\" CVPR 2019.\n\n[h] Li, Junnan, et al. \"Blip-2: Bootstrapping  language-image pre-training with frozen image encoders and large language models.\" ICML 2023.\n\n[i] Gal, Rinon, et al. \"An image is worth one word: Personalizing text-to-image generation using textual inversion.\" arxiv 2022."}]

Author rebuttal:

Meta-review: This paper was reviewed by four experts in the field. Most concerns are solved in the rebuttal and discussions. The paper finally received positive reviews with 2 Borderline Accepts and 2 Accepts. 

This paper proposes a training-free style transfer framework for sketch generation. It injects style features by modifying the K and V components in the attention layers, and uses AdaIN to interpolate between different sketch styles. 

UniTransfer, a framework controllable video concept transfer. It spatially decomposes videos into foreground, background, and motion flow, using a dual-to-single-stream DiT architecture for fine-grained control. It employs a chain-of-prompt strategy for timestep decomposition, leveraging LLMs to guide the denoising process progressively. It proposes a  dual guidance formulation to disentange the text (content) and style impact.

**strengths**
- clear motivation, well written
- a training-free simple yet effective framework, applicable to both SD 1.5 and SD XL
- good performence quantitatively and qualitatively

**weaknesses**
- insufficient experiments: missing user study, lack of evaluation on diverse styles, missing comparison on running time
- limited technical novelty considering K/V modification, AdaIN are well established methods. 

According to the reviews, the reviewers acknowledge the paper's good results and simple idea. The missing experiments are provided in the rebuttal. Following the rebuttal, all reviewers have provided positive ratings for the paper. The AC agrees with the Reviewer W249 that "The proposed method builds on many existing ideas in the literaturee, which could be seen as a limited technical novelty." but "if the model clearly outperforms previous baselines, I would consider this a minor issue". 
In the revision, the authors are suggested to incorporate the reviewers' feedback, to include the required experimental results.
Decision: Accept (poster)