Title: Text to Sketch Generation with Multi-Styles
Venue: NeurIPS 2025
Accepted: yes
Keywords: sketch genration

multi-style generation

text to image

diffusion model
Abstract: Recent advances in vision-language models have facilitated progress in sketch generation. However, existing specialized methods primarily focus on generic synthesis and lack mechanisms for precise control over sketch styles. In this work, we propose a training-free framework based on diffusion models that enables explicit style guidance via textual prompts and referenced style sketches. Unlike previous style transfer methods that overwrite key and value matrices in self-attention, we incorporate the reference features as auxiliary information with linear smoothing and leverage a style-content guidance mechanism. This design effectively reduces content leakage from reference sketches and enhances synthesis quality, especially in cases with low structural similarity between reference and target sketches. Furthermore, we extend our framework to support controllable multi-style generation by integrating features from multiple reference sketches, coordinated via a joint AdaIN module. Extensive experiments demonstrate that our approach achieves high-quality sketch generation with accurate style alignment and improved flexibility in style control. The official implementation of M3S is available at  https://github.com/CMACH508/M3S.

Text to Sketch Generation with Multi-Styles

Abstract

Recent advances in vision-language models have facilitated progress in sketch generation. However, existing specialized methods primarily focus on generic synthesis and lack mechanisms for precise control over sketch styles. In this work, we propose a training-free framework based on diffusion models that enables explicit style guidance via textual prompts and referenced style sketches. Unlike previous style transfer methods that overwrite key and value matrices in selfattention, we incorporate the reference features as auxiliary information with linear smoothing and leverage a style-content guidance mechanism. This design

effectively reduces content leakage from reference sketches and enhances synthesis quality, especially in cases with low structural similarity between reference and target sketches. Furthermore, we extend our framework to support controllable multi-style generation by integrating features from multiple reference sketches, coordinated via a joint AdaIN module. Extensive experiments demonstrate that our approach achieves high-quality sketch generation with accurate style alignment and improved flexibility in style control. The official implementation of M3S is available at https://github.com/CMACH508/M3S .

1 Introduction

Sketching, as a universal visual medium with historical roots spanning millennia, demonstrates remarkable accessibility by transcending age and cultural barriers [54]. This unique capacity to convey complex concepts through minimal strokes enables effective cross-linguistic communication, establishing it as a potent tool for ideation and conceptualization. Through historical development, sketching has evolved into a multidisciplinary practice permeating diverse domains ranging from industrial prototyping and artistic expression to educational visualization and recreational applications. This stylistic diversity, amplified by modern digital tools, introduces significant challenges for automated sketch generation systems to capture and reproduce specific artistic styles accurately.

A fundamental challenge in advancing sketch generation methods originates from the inherent difficulties in data acquisition. Unlike natural images that can be readily obtained through web scraping, high-quality sketch datasets necessitate specialized artistic expertise and substantial time investment for creation, resulting in constrained dataset scales [37, 50, 57]. While freehand sketches with rough strokes exhibit greater accessibility [7, 28, 35], including large-scale collections like QuickDraw [11], their abstract and sparse representations pose inherent limitations for training models to generate high-fidelity sketches with fine-grained details [4, 58]. Furthermore, this representational gap significantly impedes the effectiveness of captioning models [22] in producing accurate textual descriptions for sketches, thereby creating a critical bottleneck for text-conditioned sketch generation.

Recent work has explored leveraging pretrained vision-language models for sketch generation. CLIPasso [46] utilizes CLIP's cross-modal embedding space [32] to optimize Bézier curve parameters by minimizing a semantic consistency loss between rasterized sketches and natural images. Building on this idea, CLIPascene [45] introduces explicit scene decomposition to separate foreground and background elements. However, both methods depend heavily on existing image references for supervision. DiffSketcher [53] takes a step further by incorporating text-driven guidance. It combines Stable Diffusion [33] with Bézier curve optimization through score distillation sampling, enabling free-form sketch synthesis from textual prompts. Despite this advancement, existing methods [53, 18] suffer from limited control over stylistic attributes. In particular, text-based style conditioning often lacks the expressiveness and specificity needed to capture fine-grained visual traits, making it difficult to match exemplar styles precisely.

In exemplar-based style transfer, a prominent technical direction involves injecting visual features through the diffusion model's self-attention mechanisms. Representative works [3, 1, 5] propose swapping the K/V matrices derived from the reference images' denoising process into target generation. However, this direct replacement strategy proves suboptimal when handling cross-domain scenarios: The inherent discrepancy between reference and target domains induces misalignment between generated queries Q and substituted K/V features [1], leading to both content leakage and deteriorated generation quality. [63] addresses this by introducing attention distillation to align target K/V features with reference characteristics. However, this approach still suffers from excessive feature alignment that compromises output authenticity.

We propose M ultiS tyle S ketch S ynthesis (M3S), a training-free framework for generating sketches with diverse and controllable styles. To balance stylistic fidelity and content preservation, we introduce a K/V injection scheme composed of three components: (1) Hybrid attention fusion, which injects reference style features ( K ref / V ref ) with target features ( K tar / V tar ) in self-attention layers to integrate style while retaining content semantics; (2) Linear feature blending to mitigate content leakage from the reference; and (3) Separated guidance control, which divides classifier-free sampling into style and content directions, allowing flexible trade-offs between expression and

structure. This coordinated design enables high-quality, style-specific synthesis, as illustrated in the upper portion of Fig. 1.

We further investigate the feasibility of multi-style sketch generation. Sketch stylistic expression fundamentally manifests through stroke geometry and sparse texture patterns, contrasting with the dense pixel representations of natural images. Capitalizing on this characteristic, we extend M3S's feature injection to multi-style synthesis by integrating K/V features from diverse references. Additionally, drawing inspiration from [1, 15], we implement a joint AdaIN modulation on intermediate denoising steps to regulate stylistic dominance. This allows users to adjust the blending weights η between styles, for example, controlling the clown texture density in the second last row of Fig. 1.

In summary, our main contributions are as follows: (1) We propose a training-free framework for multi-style sketch generation. Key and value features from the referenced style sketches are considered auxiliary information, combined with linear smoothing and a style-content guidance mechanism, enabling the balance between style consistency and fidelity. (2) We investigate the potential of leveraging pre-trained diffusion models' prior knowledge for sketch generation with mixed styles. By introducing a joint AdaIN modulation mechanism, our approach provides users with flexible control over style generation tendencies. (3) We implement our method on Stable Diffusion v1.5 [33] and SDXL [29], with extensive experiments demonstrating the method's effectiveness, outperforming state-of-the-art approaches.

2 Related Work

Sketch Synthesis The field of sketch generation has evolved through distinct methodological paradigms. Early breakthroughs like SketchRNN [11] established sequential modeling using RNNbased encoder-decoders for single-category sketch synthesis. Subsequent work enhanced multicategory generation through structured representations: pixel-space modeling [60, 25, 41], graphbased architectures [43, 31, 59], and stroke-level analysis [24, 51]. Recent diffusion-based approaches [49, 6] improved output quality but remain constrained to coarse category-level control. Although SketchAgent [47] exploits the powerful text comprehension capability of Large Language Models (LLMs) to guide LLMs to generate sketches by using drawing rules and reference drawing sequences as prompts, the generalizability of such approaches still needs to be further explored. Parallel image-to-sketch extraction research progressed from CNN-based methods [17, 23] (requiring paired training data) to training-free techniques leveraging vision-language priors [46, 45, 53, 55]. While existing solutions address specific aspects of sketch generation, they exhibit critical limitations in simultaneous style-text controllability.

Image Style Transfer Style transfer has evolved significantly since the seminal work of Gatys et al. [10], who first demonstrated neural style transfer using Gram matrix-based feature statistics from pretrained CNNs. Subsequent approaches improved efficiency by replacing iterative optimization with feed-forward networks [19, 44], while AdaIN [15] and WCT [26] enabled real-time arbitrary style transfer through feature statistics alignment. However, these frameworks [2, 8] are less generalizable for arbitrary style transfer. Different interesting methods are proposed with the progress of the text-toimage diffusion model. InST [62] inverses a painting into corresponding textual embeddings to guide the text-to-image generative model in creating images of specific artistic appearance. B-LoRA [9] and DEADiff [30] achieve style-content separation by LoRA weight optimization and joint text-image cross-attention layers, respectively. IP-adapter [56] trained a light-weight network to incorporate style embeddings via a cross-attention mechanism. InstantStyle [48] improves the performance by injecting style features into the selective layers of the denoising UNet. CSGO [52] enhances the capacity of the style adapter by training on a curated style dataset. Based on CSGO, StyleStudio [21] proposes the cross-modal AdaIN for harmonizing style and text features. Another method is to perform K/V feature injection [1, 5] or distillation [63] of the reference image at the self-attention layers. Notably, StyleAligned [12] also leverages referenced features as auxiliary information, similar to M3S, it differs in methodology. Their method enforces distributional alignment between target and referenced features through statistical constraints. Such rigid matching can degrade performance and cause content leakage when handling sketches with large structural variance (see Section 4).

Null-text

inversion

Null-text

inversion

A sketch of a

dolphin leaping out

of the sea

Referenced Style 1

Referenced Style 2

𝑓 𝑟𝑒𝑓

1

𝑡

𝑓 𝑡𝑎𝑟

𝑡

𝑓 𝑟𝑒𝑓

2

𝑡

𝑉 𝑡𝑎𝑟

𝑡

𝐾𝑡𝑎𝑟

𝑡

𝑄𝑡𝑎𝑟

𝑡

𝐾𝑟𝑒𝑓

1

𝑡

𝑉 𝑟𝑒𝑓

1

𝑡

𝑄𝑟𝑒𝑓

1

𝑡

𝐾𝑟𝑒𝑓

2

𝑡

𝑉 𝑟𝑒𝑓

2

𝑡

𝑄𝑟𝑒𝑓

2

𝑡

Scaled Dot-

Product Attention

Self-Attention

(

𝑄𝑟𝑒𝑓

1

𝑡

,

𝐾𝑟𝑒𝑓

1

𝑡

,

𝑉 𝑟𝑒𝑓

1

𝑡

)

Self-Attention

(

𝑄𝑟𝑒𝑓

2

𝑡

,

𝐾𝑟𝑒𝑓

2

𝑡

,

𝑉 𝑟𝑒𝑓

2

𝑡

)

𝑧 𝑡

𝑡𝑎𝑟

𝑧

𝑡

𝑟𝑒𝑓

1

𝑧

𝑡

𝑟𝑒𝑓

2

Style-Content

Guidance

Joint

AdaIN

Module

𝑧 𝑡-

1

𝑡𝑎𝑟

𝐼

𝑟𝑒𝑓

2

𝐼

𝑟𝑒𝑓

1

𝜂

1

-𝜂

𝐼

𝑡𝑎𝑟

𝐾

𝑟𝑒𝑓

1

𝑡

𝐾

𝑟𝑒𝑓

1

𝑡

𝐾

𝑟𝑒𝑓

1

𝑡

𝐾

𝑟𝑒𝑓

1

𝑡

𝐾𝑡𝑎𝑟

𝑡

𝐾𝑡𝑎𝑟

𝑡

𝑄𝑡𝑎𝑟

𝑡

𝑄𝑡𝑎𝑟

𝑡

𝑉 𝑡𝑎𝑟

𝑡

𝑉 𝑡𝑎𝑟

𝑡

𝑉

𝑟𝑒𝑓

1

𝑡

𝑉

𝑟𝑒𝑓

1

𝑡

𝑉

𝑟𝑒𝑓

1

𝑡

𝑉

𝑟𝑒𝑓

1

𝑡

3 Methodology

M3S is a training-free framework for multi-style sketch synthesis that integrates textual prompts with reference style sketches through a pre-trained diffusion backbone. As illustrated in Fig. 2, our method supports single- and multi-style generation by blending style features. The single-style scenario is achieved by disabling one of the branches (lower path in Fig. 2) and setting joint AdaIN coefficients η = 1 . Below, we detail the core components and additional counter-based regular guidance for sparse and abstract freehand sketches.

3.1 Style Features Injection

Given referenced sketches I ref 1 and I ref 2 , our framework aims to synthesize the target sketch I tar through controlled feature injection. We first formalize single-style generation by integrating reference features into the denoising steps. At each timestep t ∈ ( T, ..., 0) , we extract key-value pairs ( K ref 1 , V ref 1 ) , from the self-attention layer l of the reference sketch's denoising path, while computing target features ( Q tar , K tar , V tar ) in the corresponding layer. The standard attention mechanism operates as:

Limitations of previous work. Existing approaches [1, 3, 5] achieve specified-style image generation (or style transfer) through direct feature substitution: Attention ( Q ref , K tar , V tar ) → Attention ( Q tar , K ref 1 , V ref 1 ) . However, this substitution paradigm encounters critical limitations when handling structurally divergent reference-target pairs, particularly for sketches where sparse representations amplify domain gaps. As visualized in column 2 of Fig. 3(a), such methods erroneously focus on local texture replication rather than coherent stroke synthesis, resulting in structural incoherence and artifactual patterns.

Proposed feature injection approach. As evidenced in column 4 of Fig. 3(a), our simple yet effective feature concatenation strategy alleviates these limitations. The revised attention computation is mathematically formulated as Attention ( Q tar , [ K tar K ref 1 ] , [ V tar V ref 1 ]) . However, this approach still introduces localized chaotic strokes in certain regions. For natural image synthesis, StyleAligned

Referenced

Styles

K/V Swap

Q/K AdaIN

(StyleAligned)

Ours (without

Linear Smoothing)

Ours (with Linear

Smoothing)

A sketch of a bridge over a narrow river

A sketch of a single feather falling through air

Referenced

Styles

Without counter-

based regulation

With counter-

based regulation

A sketch of a squirrel holding an acorn

A sketch of a fruit basket with fruits

(a)

(b)

[12] takes this strategy and attempts to enhance output quality through statistical alignment by modulating: Q tar = AdaIN ( Q tar , Q ref 1 ) and K tar = AdaIN ( K tar , K ref 1 ) , where the AdaIN is

Unfortunately, different from the scenario of natural images, our experiments (see Fig. 3(a), column 3) demonstrate that such rigid statistical constraints harm sketch generation quality. Instead of this constraint, the key insight underlying M3S is blending deep features from the target content into the reference K/V features, strategically trading style consistency for enhanced visual plausibility. This is realized through a linear smoothing operation with the hyperparameter λ ∈ [0 , 1] :

Increasing λ generally enhances aesthetic quality and text alignment. However, as our base model is trained on natural images, excessively high λ risks style degradation and naturalistic outputs over sketches. To synthesize sketches combining styles from both I ref 1 and I ref 2 , we extend Eq.(3) by simply concatenate additional features ˆ K ref 2 = λK tar +(1 -λ ) K ref 2 and ˆ V ref 2 = λV tar +(1 -λ ) V ref 2 . The attention can be written as Attention ( Q tar , [ K tar , ˆ K ref 1 , ˆ K ref 2 , ] T , [ V tar , ˆ V ref 1 , ˆ V ref 2 , ] T ) .

3.2 Control the Style Tendency

Inspired by [1], we introduce AdaIN modulation for latent noise images to address the color distribution shift problem. This process for single style can be formalized as z tar t = AdaIN ( z tar t , z ref 1 t ) , where z tar t is the latent target image in denoising time step t and z ref 1 t is obtained by the null-text inversion technology [16]. Next, we discuss the impact of the modulation on sketch style. Assuming a sketch is represented in a bitmap way (0 for strokes, 1 for background), referenced style sketches with dense strokes (e.g., >50% black pixels) exhibit low mean values. AdaIN modulation consequently biases generated sketches toward a lower mean value µ , yielding detailed outputs. Conversely, abstract sketches with sparse strokes demonstrate higher µ , producing more minimalist results. Motivated by this analysis, we introduce a Joint AdaIN module for multi-style tendency control:

where η ∈ [0 , 1] is the tendency parameter. Notably, the synthesized results maintain multi-style characteristics even at parameter extremes ( η = 0 or η = 1 ), as the self-attention calculation incorporates more than a single stylistic feature.

3.3 Style-Content Guidance

While text-to-image diffusion models excel at natural image synthesis, their inherent bias toward photorealistic outputs poses challenges for stylized sketch generation. Conventional classifier-free guidance (CFG) [14] combined with feature injection (Section 3.1) and AdaIN modulation (Section 3.2) struggles to maintain stylistic consistency due to fundamental domain discrepancies between natural images and sketches. To address this, we introduce a null-text conditioned style guidance term that establishes dual control pathways:

where ϵ × θ ( · ) denotes the noise predicted with feature injection (Section 3.1), ω 1 and ω 2 are the content and style guidance scales, respectively. The parameters ω 1 and ω 2 require careful calibration in practice to achieve an optimal balance between style and content. Excessively high values of ω 2 may compromise text-image alignment, while insufficiently low values result in uncontrollable stylistic expression in synthesized sketches.

3.4 Counter-based Regulation Guidance

To mitigate potential artifacts in abstract sketch generation with SD v1.5 [33] (Fig. 3(b)), we first apply Tweedie's formula [20] to estimate the denoised latent representation z tar 0 | t = z tar t - √ 1 -¯ α t ˜ ϵ t √ ¯ α t [13, 40], which is decoded to image I 0 | t tar . Subsequently, we extract directional gradients through Sobel operators [39]: grad x = S x ∗ I 0 | t tar , grad y = S y ∗ I 0 | t tar , where ∗ denotes convolution. Then, we calculate the regular term loss and optimize the latent representations as

The updated z tar 0 | t is subsequently applied to calculate z tar t -1 with DDIM step [40]. In practice, we empirically set γ = 60 as the default and clamp grad x and grad y within [ -0 . 001 , 0 . 001] to prevent over-amplification of gradient magnitudes that could degrade generation quality.

4 Experiments

Dataset and Metrics We evaluate different methods for single-style referenced generation on six diverse sketch datasets encompassing professional, amateur, and abstract styles: four professional styles from 4skst [37] (Styles 1-4, artist-drawn with referenced images), a web-collected diverse style set (Style 5, 20 sketches from open-source platforms), and 50 abstract freehand sketches from Sketchy [35] (Style 6). For systematic testing, we generate 50 textual prompts via DeepSeek [27] using the template "A sketch of ...", pairing each prompt with a randomly selected referenced sketch. To evaluate the performance of M3S and baselines, CLIP-T [32] is used to measure the alignment between generated sketches and prompts. For style consistency, we use the similarity between referenced and target images with the extracted features from DINO [61]. Similar to [1], the distance of Gram matrices calculated from VGG [38] is also considered.

Implementation Details We implement M3S on both Stable Diffusion v1.5 [33] and SDXL [29], with all visual results in this paper generated by M3S (SD v1.5) unless otherwise specified (more M3S (SDXL)'s results are in the Appendix). For M3S (SD v1.5), we configure ω 1 = 15 , ω 2 = 15 and λ = 0 . 1 Styles 1-5, while using ω 1 = 15 , ω 2 = 25 and λ = 0 . 05 for Style 6. Similar to [1], feature injection is applied to self-attention layers in the UNet decoder where feature map resolutions are 32 × 32 and 64 × 64 . For M3S (SDXL), we set ω 1 = 15 , ω 2 = 15 and λ = 0 . 1 for Styles 1-5, adjusting to ω 1 = 7 . 5 , ω 2 = 20 and λ = 0 . 05 for Style 6. The specific injection layers are detailed in the Appendix. The ω 2 parameter linearly increases from ω 2 / 3 to ω 2 throughout the denoising process in both implementations. We applied DDIM [40] to sample the target sketches with 100 steps.

Ours (SDXL)

Ours (SD v1.5)

InstantStyle

Modulation

StyleStudio

CSGO

Attention-

Distillation

StyleAligned

Referenced

Styles

A sketch of a

cat sitting on a

cup

A sketch of a

crow perched

on a fence

Prompts

A sketch of a

butterfly with

plain wings

A sketch of a

portrait of

naruto

A sketch of a

fruit basket

with fruits

A sketch of a

fox standing

on a log

A sketch of a

lighthouse by

the ocean

A sketch of a

hanging

lantern at

night

TTC

W

X

色

CD

?

Baselines We compare with state-of-the-art (SOTA) methods spanning diverse technical paradigms: StyleAligned [12] (self-attention feature injection), RB-Modulation [34] (multi-attention aggregation), AttentionDisillation [63], and methods leveraging specialized style adapters - InstantStyle [48], CSGO [52], and StyleStudio [21]. These methods are primarily designed for single-reference scenarios. All baselines are implemented with their open-source codes.

4.1 Qualitative Analysis

In Fig. 4, we illustrated the qualitative synthesized results in various styles with different methods. Our method effectively captures referenced style sketch attributes, including stroke thickness, pixel density, and luminance levels, to achieve zero-shot text-aligned sketch synthesis. Compared to StyleAligned [12] and AttentionDistillation [63], which preserve stylistic fidelity at the cost of content leakage (e.g., the fox erroneously positioned on a car in the third to last column) and chaotic stroke patterns (visible in AttentionDistillation's first three columns), our M3S framework maintains strict content-semantic consistency while eliminating structural artifacts. These results demonstrate that excessive feature constraints may degrade sketch quality, particularly in artist-style sketches. Although methods like InstantStyle [48] and CSGO [52] generate aesthetically plausible results aligned with textual prompts, they exhibit discernible deviations from referenced styles and constrained style diversity compared to our approach. This phenomenon likely stems from these

Reference 1

Reference 2

Reference 1

Reference 2

Reference 1

Reference 2

A sketch of

A sketch of a

woman in a

cheongsam

A sketch of a

Nike sneaker

η

= 0

η

= 0.25

η

= 0.5

η

= 0.75

η

= 1

MKE

methods' primary focus on natural image style transfer, where their feature injection mechanisms predominantly operate in text space rather than visual space.

The multi-style fusion capability is partially illustrated through specific examples in Fig. 5, and there are some interesting cases: (1) The apple sketch (top-left) fuses distinct stylistic attributes, clear black contours from one reference, and localized texturing patterns from another. (2) The birthday cake synthesis (top-right). When fusing an abstract freehand butterfly sketch and a precisely structured car drawing, the result inherits the butterfly's stroke coloration and the car's orderly linework. Conversely, the cake's contours exhibit amateurish characteristics when substituting the car reference with other sketches. These examples illustrate our framework's capacity to selectively blend stylistic elements across references while preserving domain-appropriate stroke characteristics. Our method also enables flexible style interpolation through the Joint AdaIN module, as shown at the bottom of Fig. 5. These interpolated results may be able to stimulate the user's creativity. For instance, increasing η progressively enhances the definition of Messi's beard strokes.

4.2 Quantitative Analysis

Table 1 reports the quantitative results of different methods. Under default parameter settings, our M3S (SDXL) achieves the best average CLIP score of 0.3514, demonstrating superior text alignment. While the DINO score and VGG style loss slightly trail AttentionDistillation [63], this reflects our method's balanced style-content trade-off. To validate flexibility, we provide additional results with style-oriented parameters (denoted as Ours (SDXL ∗ ) in the table), where reducing content-guidance ω 1 and increasing style-guidance ω 2 yields style metrics comparable to AttentionDistillation while slightly retaining CLIP score advantages. This highlights M3S's user-adjustable controllability for subjective style-content balancing. Notably, methods like StyleStudio [21] produce high-quality images but exhibit lower CLIP scores, likely due to their outputs resembling natural images over

sketches. Similarly, lower CLIP scores in Style 3 may stem from dense black patches in references, whereas Style 4 shows the opposite trend. Each sketch takes about 40 seconds (M3S (SD v1.5)) and 70 seconds (M3S (SDXL)) on an A100 40GB GPU.

We conducted a human preference assessment via structured questionnaires to evaluate text-tosketch synthesis performance. Each questionnaire contained: 1) Six randomly selected sets of generated sketches. 2) Eight anonymized outputs per set from different models under identical prompts and reference styles. 3) Evaluation criteria: Comprehensive assessment across three dimensions-text alignment, style consistency, and generation quality. Participants ranked results on a scale of 1-8 (8=optimal). From 58 submissions (avg. completion: 4m16s), we excluded 14 invalid responses (<60s completion/missing rankings), retaining 44 validated questionnaires. The results are shown on Table 2. M3S (SDXL) achieved the highest average score (6.19), and M3S (SD v1.5) secured a strong performance (5.44). We evaluate the statistical significance by rank tests, revealing that M3S (SD v1.5) significantly outperforms all baseline methods except InstantStyle (p-value= 0 . 26 ). When we align the backbone with InstantStyle (i.e., SDXL), our M3S (SDXL) demonstrates superiority over it (p-value= 1 . 06 × 10 -5 ).

The quantitative results of multi-style generation are shown in Table 3. We generate outputs for each prompt by randomly selecting two reference sketches from the Style 5 (S5) dataset. To specifically validate generation performance with significantly distinct reference styles, we conducted an additional experiment set pairing one randomly selected S5 image with one randomly chosen image from the QuickDraw (QD) dataset [11] per prompt. When the references exclusively originate from S5, M3S maintains text alignment comparable to single-style generation. For style consistency, DINOref1 exhibits a positive correlation, while DINO-ref2 shows a negative correlation. At boundary conditions ( η = 0 or 1 ) of multi-style sketch generation, only one style participates in AdaIN modulation for image generation. For the two reference styles, replacing the pair of S5-S5 with QD-S5 reduces style consistency for both implementations (i.e., SD v1.5 and SDXL), though M3S (SD v1.5) demonstrates superior robustness. Crucially, M3S (SDXL) struggles to effectively utilize QD's abstract features, evidenced by significantly lower scores of DINO-ref1 than DINO-ref2 in QD-S5 pairs. This limitation stems from SDXL's high-fidelity optimization [29] - its user-tested superiority over SD v1.5 creates inherent incompatibility with low-quality, abstract datasets like QD.

4.3 Ablation Study

We conduct the ablation study on styles 1-4 with SD V1.5 as the backbone. To evaluate the effectiveness of our feature injection approach, Table 4 reports the average scores across these styles. Aesthetic metric [36] is included to reflect the effect more comprehensively. When employing basic K/V Swap, the output maintains excellent style consistency with a DINO score of 0.8109, but its low CLIP score reflects poor alignment between generated sketches and textual prompts.

By incorporating reference features as auxiliary inputs (i.e., λ = 0 ), the CLIP metric improves by 9.26%, accompanied by an acceptable reduction in DINO. However, the aesthetic score of generated images under this configuration drops to 4.7952, significantly lower than the reference style average of 5.0549. Increasing λ mitigates aesthetic degradation and improves text alignment, but further erodes style fidelity. Beyond a critical λ , CLIP gains diminish while style loss intensifies, leading to our experimentally determined default of λ = 0 . 1 as an optimal compromise. Fig. 6 demonstrates the impact of content guidance ( ω 1 ) and style guidance ( ω 2 ) on generation results. When ω 1 is fixed, increasing ω 2 reduces the number of intersections in curvilinear textures on the deer's body,

𝜔

1

= 15

𝜔

2

= 0

𝜔

1

= 15

𝜔

2

= 5

𝜔

1

= 15

𝜔

2 = 10

𝜔

1

= 15

𝜔

2 = 15

𝜔

1

= 3

𝜔

2 = 15

𝜔

1

= 6

𝜔

2 = 15

𝜔

1

= 9

𝜔

2 = 15

driving the output closer to the reference style. Conversely, enhancing ω 1 under a fixed ω 2 improves the visual quality of the generated deer. More ablation results are in the Appendix.

5 Conclusions

We propose a novel training-free framework named M3S for zero-shot sketch synthesis blending different styles. The key insight in our method is to take the referenced features as auxiliary information and modulate the latent noise images with the joint AdaIN module. This modulated approach enables the user to control the style tendency. Additionally, the style-content direction guidance provides the flexibility to balance the fidelity and style consistency. A direction for future research and the functionality not included for now is achieving localized style control, where users can explicitly assign specific styles to particular regions rather than relying on the model's automatic style assignment. This capability would further assist and inspire artistic creation.

References

Yuval Alaluf, Daniel Garibi, Or Patashnik, Hadar Averbuch-Elor, and Daniel Cohen-Or. Crossimage attention for zero-shot appearance transfer. In ACMSIGGRAPH 2024 Conference Papers , pages 1-12, 2024.

Amirsaman Ashtari, Chang Wook Seo, Cholmin Kang, Sihun Cha, and Junyong Noh. Reference based sketch extraction via attention mechanism. ACM Transactions on Graphics (TOG) , 41(6):1-16, 2022.

Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free mutual self-attention control for consistent image synthesis and editing. In Proceedings of the IEEE/CVF international conference on computer vision , pages 22560-22570, 2023.

Yajing Chen, Shikui Tu, Yuqi Yi, and Lei Xu. Sketch-pix2seq: a model to generate sketches of multiple categories. arXiv preprint arXiv:1709.04121 , 2017.

Jiwoo Chung, Sangeek Hyun, and Jae-Pil Heo. Style injection in diffusion: A training-free approach for adapting large-scale diffusion models for style transfer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 8795-8805, 2024.

Ayan Das, Yongxin Yang, Timothy Hospedales, Tao Xiang, and Yi-Zhe Song. Chirodiff: Modelling chirographic data with diffusion models. In The Eleventh International Conference on Learning Representations , 2023.

Mathias Eitz, James Hays, and Marc Alexa. How do humans sketch objects? ACM Transactions on graphics (TOG) , 31(4):1-10, 2012.

Noa Fish, Lilach Perry, Amit Bermano, and Daniel Cohen-Or. Sketchpatch: Sketch stylization via seamless patch-level synthesis. ACM Transactions on Graphics (TOG) , 39(6):1-14, 2020.

Yarden Frenkel, Yael Vinker, Ariel Shamir, and Daniel Cohen-Or. Implicit style-content separation using b-lora. In European Conference on Computer Vision , pages 181-198. Springer, 2024.

Leon A Gatys, Alexander S Ecker, and Matthias Bethge. Image style transfer using convolutional neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 2414-2423, 2016.

David Ha and Douglas Eck. A neural representation of sketch drawings. arXiv preprint arXiv:1704.03477 , 2017.

Amir Hertz, Andrey Voynov, Shlomi Fruchter, and Daniel Cohen-Or. Style aligned image generation via shared attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4775-4785, 2024.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems , 33:6840-6851, 2020.

Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598 , 2022.

Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. In Proceedings of the IEEE international conference on computer vision , pages 1501-1510, 2017.

Inbar Huberman-Spiegelglas, Vladimir Kulikov, and Tomer Michaeli. An edit friendly ddpm noise space: Inversion and manipulations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 12469-12478, 2024.

Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 1125-1134, 2017.

Ajay Jain, Amber Xie, and Pieter Abbeel. Vectorfusion: Text-to-svg by abstracting pixel-based diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 1911-1920, 2023.

Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14 , pages 694-711. Springer, 2016.

Kwanyoung Kim and Jong Chul Ye. Noise2score: tweedie's approach to self-supervised image denoising without clean images. Advances in Neural Information Processing Systems , 34:864-874, 2021.

Mingkun Lei, Xue Song, Beier Zhu, Hao Wang, and Chi Zhang. Stylestudio: Text-driven style transfer with selective control of style elements. arXiv preprint arXiv:2412.08503 , 2024.

Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning , pages 19730-19742. PMLR, 2023.

Mengtian Li, Zhe Lin, Radomir Mech, Ersin Yumer, and Deva Ramanan. Photo-sketching: Inferring contour drawings from images. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV) , pages 1403-1412. IEEE, 2019.

Tengjie Li, Shikui Tu, and Lei Xu. Sketchedit: editing freehand sketches at the stroke-level. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence , pages 4461-4469, 2024.

Tengjie Li, Sicong Zang, Shikui Tu, and Lei Xu. Lmser-pix2seq: Learning stable sketch representations for sketch healing. Computer Vision and Image Understanding , 240:103931, 2024.

Yijun Li, Chen Fang, Jimei Yang, Zhaowen Wang, Xin Lu, and Ming-Hsuan Yang. Universal style transfer via feature transforms. Advances in neural information processing systems , 30, 2017.

Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437 , 2024.

Kushin Mukherjee, Holly Huey, Xuanchen Lu, Yael Vinker, Rio Aguina-Kang, Ariel Shamir, and Judith Fan. Seva: Leveraging sketches to evaluate alignment between human and machine visual abstraction. Advances in Neural Information Processing Systems , 36:67138-67155, 2023.

Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952 , 2023.

Tianhao Qi, Shancheng Fang, Yanze Wu, Hongtao Xie, Jiawei Liu, Lang Chen, Qian He, and Yongdong Zhang. Deadiff: An efficient stylization diffusion model with disentangled representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8693-8702, 2024.

Yonggang Qi, Guoyao Su, Pinaki Nath Chowdhury, Mingkang Li, and Yi-Zhe Song. Sketchlattice: Latticed representation for sketch manipulation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 953-961, 2021.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , pages 8748-8763. PmLR, 2021.

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. Highresolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10684-10695, 2022.

Litu Rout, Yujia Chen, Nataniel Ruiz, Abhishek Kumar, Constantine Caramanis, Sanjay Shakkottai, and Wen-Sheng Chu. Rb-modulation: Training-free personalization of diffusion models using stochastic optimal control, 2024.

Patsorn Sangkloy, Nathan Burnell, Cusuh Ham, and James Hays. The sketchy database: learning to retrieve badly drawn bunnies. ACM Transactions on Graphics (TOG) , 35(4):1-12, 2016.

Christoph Schuhmann. Improved aesthetic predictor, 2022.

Chang Wook Seo, Amirsaman Ashtari, and Junyong Noh. Semi-supervised reference-based sketch extraction using a contrastive learning framework. ACM Transactions on Graphics (TOG) , 42(4):1-12, 2023.

Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 , 2014.

Irwin Sobel, Gary Feldman, et al. A 3x3 isotropic gradient operator for image processing. a talk at the Stanford Artificial Project in , 1968:271-272, 1968.

Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations , 2021.

Jifei Song, Kaiyue Pang, Yi-Zhe Song, Tao Xiang, and Timothy M Hospedales. Learning to sketch with shortcut cycle consistency. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 801-810, 2018.

Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations , 2021.

Guoyao Su, Yonggang Qi, Kaiyue Pang, Jie Yang, and Yi-Zhe Song. Sketchhealer a graph-tosequence network for recreating partial human sketches. In Proceedings of The 31st British Machine Vision Virtual Conference (BMVC 2020) , pages 1-14. British Machine Vision Association, 2020.

Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Improved texture networks: Maximizing quality and diversity in feed-forward stylization and texture synthesis. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 6924-6932, 2017.

Yael Vinker, Yuval Alaluf, Daniel Cohen-Or, and Ariel Shamir. Clipascene: Scene sketching with different types and levels of abstraction. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 4146-4156, 2023.

Yael Vinker, Ehsan Pajouheshgar, Jessica Y Bo, Roman Christian Bachmann, Amit Haim Bermano, Daniel Cohen-Or, Amir Zamir, and Ariel Shamir. Clipasso: Semantically-aware object sketching. ACM Transactions on Graphics (TOG) , 41(4):1-11, 2022.

Yael Vinker, Tamar Rott Shaham, Kristine Zheng, Alex Zhao, Judith E Fan, and Antonio Torralba. Sketchagent: Language-driven sequential sketch generation. arXiv preprint arXiv:2411.17673 , 2024.

Haofan Wang, Matteo Spinelli, Qixun Wang, Xu Bai, Zekui Qin, and Anthony Chen. Instantstyle: Free lunch towards style-preserving in text-to-image generation. arXiv preprint arXiv:2404.02733 , 2024.

Qiang Wang, Haoge Deng, Yonggang Qi, Da Li, and Yi-Zhe Song. Sketchknitter: Vectorized sketch generation with diffusion models. In The Eleventh International Conference on Learning Representations , 2023.

Xiaogang Wang and Xiaoou Tang. Face photo-sketch synthesis and recognition. IEEE transactions on pattern analysis and machine intelligence , 31(11):1955-1967, 2008.

Xin Wang, Tengjie Li, Sicong Zang, Shikui Tu, and Lei Xu. Self-supervised learning for enhancing spatial awareness in free-hand sketch. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence , pages 5117-5125, 2024.

Peng Xing, Haofan Wang, Yanpeng Sun, Qixun Wang, Xu Bai, Hao Ai, Renyuan Huang, and Zechao Li. Csgo: Content-style composition in text-to-image generation. arXiv preprint arXiv:2408.16766 , 2024.

Ximing Xing, Chuang Wang, Haitao Zhou, Jing Zhang, Qian Yu, and Dong Xu. Diffsketcher: Text guided vector sketch synthesis through latent diffusion models. Advances in Neural Information Processing Systems , 36:15869-15889, 2023.

Peng Xu, Timothy M Hospedales, Qiyue Yin, Yi-Zhe Song, Tao Xiang, and Liang Wang. Deep learning for free-hand sketch: A survey. IEEE transactions on pattern analysis and machine intelligence , 45(1):285-312, 2022.

Rui Yang, Xiaojun Wu, and Shengfeng He. Mixsa: Training-free reference-based sketch extraction via mixture-of-self-attention. IEEE Transactions on Visualization and Computer Graphics , 2024.

Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.06721 , 2023.

Ran Yi, Yong-Jin Liu, Yu-Kun Lai, and Paul L. Rosin. Apdrawinggan: Generating artistic portrait drawings from face photos with hierarchical gans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , June 2019.

Sicong Zang, Shikui Tu, and Lei Xu. Controllable stroke-based sketch synthesis from a self-organized latent space. Neural Networks , 137:138-150, 2021.

Sicong Zang, Shikui Tu, and Lei Xu. Linking sketch patches by learning synonymous proximity for graphic sketch representation. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 37, pages 11096-11103, 2023.

Sicong Zang, Shikui Tu, and Lei Xu. Self-organizing a latent hierarchy of sketch patterns for controllable sketch synthesis. IEEE Transactions on Neural Networks and Learning Systems , 2023.

Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and HeungYeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605 , 2022.

Yuxin Zhang, Nisha Huang, Fan Tang, Haibin Huang, Chongyang Ma, Weiming Dong, and Changsheng Xu. Inversion-based style transfer with diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10146-10156, 2023.

Yang Zhou, Xu Gao, Zichong Chen, and Hui Huang. Attention distillation: A unified approach to visual characteristics transfer. arXiv preprint arXiv:2502.20235 , 2025.

Appendix

The appendix is organized into several sections, including more analysis and additional details. These topics are as follows:

In Section A, we provide preliminaries about diffusion model.

In Section B, more implementation details of our M3S are provided.

In Section C, we analyze the effectiveness of the regular term for abstract and sparse sketch generation.

In Section D, we present the influence of the content guidance ω 1 and the style guidance ω 2 .

In Section E, we discuss the effectiveness of linear smoothing for style injection.

In Section F, we compare other generative model for sketch synthesis.

In Section G, we provide more synthesized results of M3S (SDXL).

In Section H, we discuss the limitations of M3s.

In Section I, we discuss the potential social impact of our method.

In Section J, we show the selected images of Style 5.

In Section K, we provide the prompts used for qualitative analysis.

In Section L, we present the prompts for Fig. 5's multi-style generation.

A Preliminaries

A.1 Denoising Diffusion Probabilistic Models (DDPM)

Denoising Diffusion Probabilistic Models (DDPMs) [13] establish a generative framework that learns to model the data distribution q data ( x 0 ) through two complementary phases:

Forward Diffusion Process. The forward process systematically perturbs data samples through progressive noise injection. Given initial data x 0 ∼ q data ( x 0 ) , it constructs a Markov chain of latent variables { x t } T t =1 by gradually adding Gaussian noise according to a predefined schedule { β t } T t =1 ∈ (0 , 1) :

This transforms the complex data distribution into an isotropic Gaussian distribution q ( x T ) ≈ N ( 0 , I ) .

Reverse Denoising Process. The reverse process learns to invert the diffusion trajectory by iteratively denoising from x T ∼ N ( 0 , I ) . Since the true reverse transition q ( x t -1 | x t ) depends on the intractable data distribution q data ( x 0 ) , DDPMs approximate it through a learned conditional Gaussian:

Training Objective. Instead of direct mean prediction, DDPMs adopt a noise prediction parameterization. For timestep t uniformly sampled from { 1 , ..., T } , the network ϵ θ predicts the injected noise through:

where α t := 1 -β t and ¯ α t := ∏ t s =1 α s . The denoised mean is then derived as:

During inference, DDPMs sample from p θ ( x t -1 | x t ) iteratively from t = T to t = 1 . The complete derivation and connections to stochastic differential equations are detailed in [42].

Default

Default without layer 71

Default without layer 69

Layers [69, 71]

All layers

Layers [65,67,69, 71]

Referenced styles

A sketch of a pine tree on a hill

A sketch of a sailboat floating on calm water

B More Implementation Details

To eliminate the possible localized shadows in the generated sketches, we followed the practice of [55] and performed a brightening operation on the images. Specifically, we set pixels with values exceeding 0.7 to 1, where the pixel value range is normalized between -1 and 1.

Our M3S (SDXL) selects some self-attention layers from the upsample UNet branch to perform the feature injection. We first enumerate all attention layers in the decoder, where odd-numbered indices correspond to self-attention layers. In practice, layers with indices [1,9,17,25,33,41,49,57,69,71] are selected for reference style information injection. The examples in Fig. 7 demonstrate the critical role of the final self-attention layer in style control. When feature injection is disabled at layer 71, the generated images overemphasize textual prompts, resulting in excessive color patches that deviate from the reference style. Conversely, injecting features across all self-attention layers achieves exceptional style consistency but compromises text alignment (e.g., excessive folds in generated sailboat sails). This indicates our layer selection strategy inherently balances style-content trade-offs. Users can adopt the default configuration or customize layer choices based on visual examples to meet specific needs.

C Analysis of the Regular Term

The regularization term in Eq. (6) primarily serves to suppress shadows and enhance stroke definition. As demonstrated in Fig. 8, increasing hyperparameter γ progressively diminishes shadow regions while accentuating strokes, albeit with unintended over-sharpening that introduces stroke unevenness. Empirical analysis reveals optimal performance when γ resides within [40, 60], balancing shadow removal and stroke smoothness. Applying this technique to enhance abstract sketches introduces approximately 5 seconds of additional processing time per image.

D Analysis of the Style-Content guidance

Fig. 9 demonstrates the effects of content guidance ( ω 1 ) and style guidance ( ω 2 ) through professional and abstract style examples. For the professional style (left): (1) All configurations yield textcompliant sketches. (2) High ω 1 values (20/25) introduce extraneous pens, likely due to insufficient style guidance during early denoising (note ω 2 's linear scheduling in Paragraph Implementation Details). (3) Consistent with Fig. 6, we observe that higher ω 1 enhances aesthetics (e.g., streamlined deer faces) and increased ω 2 improves style adherence (rule-based non-intersecting fur strokes). (4) Balancing trade-offs: At ω 1 = 5 , elevating ω 2 causes structural distortion (deer torso deformation at ω 2 = 20 ). This is mitigated by increasing ω 1 , motivating our default ω 1 = 15 , ω 2 = 15 .

For abstract styles (right): (1) ω 1 = 5 , ω 2 = 5 produces over-detailed faces inconsistent with the referenced style sketch. (2) Higher ω 2 eliminates excessive details but weakens "yoga" characteristics. We therefore set ω 1 = 15 , ω 2 = 25 for abstract cases, accepting marginal quality degradation to prevent artifacts like striped pants (visible at ω 1 = 15 , ω 2 = 15 ).

A sketch of a monkey

eating a peach

A sketch of a rabbit

eating a burger

A sketch of James

Bond in a tuxedo and

holding a gun

A sketch of Trump

playing guitar

γ

= 0

γ

= 20

γ

= 40

γ

= 60

γ

= 80

S

7.00m

𝜔

1

= 5

𝜔

2

= 5

𝜔

2

= 10

𝜔

2

= 15

𝜔

2

= 20

𝜔

2

= 25

𝜔

1

= 10

𝜔

1

= 15

𝜔

1

= 20

𝜔

1

= 25

Style

Style

𝜔

2

= 5

𝜔

2

= 10

𝜔

2

= 15

𝜔

2

= 20

𝜔

2

= 25

𝜔

1

= 5

𝜔

1

= 10

𝜔

1

= 15

𝜔

1

= 20

𝜔

1

= 25

米

E Analysis of Linear Smoothing for Feature Injection

To further analyze the impact of linear smoothing parameter λ , Fig. 10 visualizes generation results under varying λ values, while Table 5 extends the quantitative analysis from Table 4 with complete metrics. Key observations include: (1) λ = 0 : Excessive focus on reference style introduces chaotic strokes that compromise text alignment (e.g., window artifacts in row 2). (2) λ ∈ (0 , 0 . 25) : Progressive artifact reduction improves visual cleanliness (row 3 fox torso refinement) while retaining core style attributes. (3) λ = 0 . 25 : Incipient style degradation manifests as faint roof lines contradicting reference patterns (row 2). (4) λ = 1 : Complete style disengagement yields naturalistic images lacking artistic stylization. Table 5 quantitatively corroborates these findings: decreasing DINO

λ

= 0

Styles

λ

= 0.05

λ

= 0.1

λ

= 0.15

λ

= 0.2

λ

= 0.25

λ

= 1

A sketch of a bridge over a narrow river

A sketch of a small house with two windows

A sketch of a fox standing on a log

A sketch of a vintage camera on a tripod

scores and improving aesthetic metrics with increasing λ , reflecting the inherent trade-off between style preservation and content alignment.

F Comparisons with More Methods

While the main text focuses on style-specific generation methods, Fig. 11 provides a comparative analysis with broader sketch synthesis approaches. Implementations of CLIPasso [46] and DiffSketcher [53] follow their official codebases, requiring over 3 minutes per sketch generation. We observe that CLIPasso, DiffSketcher, and SketchRNN [11] produce stylistically homogeneous outputs. Notably, CLIPasso's reliance on foreground extraction networks leads to chaotic strokes when segmentation fails (e.g., indistinguishable floral patterns). Although Stable Diffusion v1.5 [33] exhibits style diversity, its outputs demonstrate uncontrolled style variation and struggle to produce sketches with white backgrounds and black strokes. Our method effectively addresses these limitations through controlled style injection while maintaining superior visual quality. When using DiffSketcher's outputs as reference styles, M3S successfully emulates this style (Fig.11-bottom), demonstrating seamless integration with specialized models.

G Generated Sketches by M3S (SDXL)

Compared to SD v1.5, SDXL employs a significantly larger denoising network architecture and enhances the dimensionality of text-conditioning embeddings. In the main text, we provide several

Content images for CLIPasso and DiffSketcher

CLIPasso

DiffSketcher

Stable Diffusion v1.5

SketchRNN

Ours

Referenced styles

公么

(E)

?

results synthesized by M3S (SD v1.5), and we illustrate generated sketches by the more powerful M3S (SDXL).

Benefiting from SDXL's superior text-to-image synthesis capabilities, our M3S framework achieves precise text-aligned sketch generation across diverse artistic styles, even for complex compositional prompts like "A sketch of a cyberpunk-style cat with mechanical limbs" , while maintaining strict adherence to reference style characteristics, as shown in Fig. 12. Fig. 13 illustrates the results of multi-style generation.

M3S (SDXL) effectively integrates styles from dual reference images through its joint AdaIN modulation mechanism for style tendency control. The mirror downside is that the M3S (SDXL) is less than

perfect in the style migration of local details. For instance, the elephant contour in row 2 demonstrates insufficient adoption of the left reference's segmented-line style, where discrete strokes appear less pronounced. This discrepancy likely stems from SDXL's architectural advancements: deeper network layers, augmented attention modules, and heightened model complexity. Notably, these modifications simultaneously enhance text-alignment precision, particularly for prompts requiring fine-grained semantic grounding.

M3S (SDXL) preserves the intrinsic diversity of diffusion models - generating distinct sketches from identical textual prompts and reference styles through stochastic initial noise sampling, as demonstrated in Fig. 14. This stochasticity manifests across multiple dimensions: (1) Form variation. Divergent animal pose. (2) Facial Articulation. Unique combinations of facial features and hairstyles. (3) Compositional novelty. Alternative spatial arrangements of scene elements. Such multi-faceted diversity provides artists with rich creative inspiration to create sketches.

H Limitation

Fig. 15 Although our M3S demonstrates superior performance in most scenarios, certain limitations persist. As shown in the left panel of Fig. 15, when reference sketches are excessively sparse with content concentrated in small image regions, M3S struggles to generate complete and clear sketches. This stems from two factors: (1) Extreme sparsity hinders effective style feature extraction for guiding generation, and (2) Higher pixel intensity averages in such references disrupt AdaIN modulation, limiting pixel availability for text-aligned sketch synthesis. We propose a potential mitigation strategy (Fig.15-right): Enhancing reference images via zooming and replication-based padding to increase pixel density. While this alleviates the issue partially, the augmentation process inevitably alters original stylistic attributes (e.g., sparsity distribution and stroke thickness). Consequently, fully resolving this limitation necessitates further investigation into non-destructive reference adaptation methods.

I Social Impact

As illustrated in Fig. 4 and Fig. 12, M3S enables high-quality artist-style sketch generation with an exemplar. It takes only a few tens of seconds to produce work that would take a human hours. Meanwhile, as shown in Fig. 5 and Fig. 13, our novel multi-style sketch generation technology and style preference control can provide users with more creative inspiration.

J Selected Style Images of Style 5

As illustrated in Fig. 16, we curated sketches spanning diverse artistic styles, including portraiture, scalar vector graphics, meticulous brushwork (gongbi), minimalist aesthetics, and detailed rendering styles, among others.

K Prompts for Quantitative Experiments

The textual prompts for quantitative evaluation cover a wide range of common real-life scenarios and categories, including animals, landscapes, vehicles, and daily objects etc. Specific prompts are as follows:

a sketch of a sailboat floating on calm water

a sketch of a pine tree on a small hill

a sketch of a cat sitting inside a teacup

a sketch of a hot air balloon high over mountains

a sketch of a dragon flying in the sky, full body

a sketch of a bicycle leaning against a brick wall

a sketch of a small house with two windows

a sketch of a fruit basket with fruits

Referenced

Styles

A sketch of a bunny

A sketch of a boy

holding an umbrella

A sketch of a

cyberpunk-style cat

with mechanical

limbs

A sketch of a fox

sitting on the sofa

A sketch of a small

house on the

mountain

A sketch of a church

?

见

D

NLI

山

a sketch of a portrait of naruto

a sketch of enchanted forest with glowing mushrooms

a sketch of James Bond in a tuxedo and holding a gun

a sketch of a man performing tai chi

a sketch of a butterfly with plain wings

a sketch of a campfire under starry sky

a sketch of sydney opera house

a sketch of a bridge over a narrow river

A sketch of a modern city

A sketch of an elephant

A sketch of a coffee mug

A sketch of a knight holding a sword

A sketch of a frog

η

= 0

η

= 0.25

η

= 0.5

η

= 0.75

η

= 1

Referenced

Style 1

Referenced

Style 2

CZZZZ

<

O

A sketch of spiderman

a sketch of a robot holding a flower

a sketch of two white bunnies

a sketch of a lighthouse by the ocean

a sketch of an owl perched on a car

a sketch of a single feather falling through air

a sketch of a pair of glasses on an open book

a sketch of a steaming coffee cup on a saucer

a sketch of a crescent moon with one star

Referenced

Styles

a sketch of a fox standing on a log

a sketch of an empty swing hanging from a tree

a sketch of a vintage camera on a tripod

a sketch of a seashell on smooth sand

a sketch of a paper airplane mid-flight

a sketch of a single rose in a slim vase

a sketch of a mountain peak piercing clouds

a sketch of a deer standing in a meadow

A sketch of

House

A sketch of a

winding

staircase in a

tower

Referenced

styles

Generated

results

Augmented

referenced styles

Generated

results

4

D

At6

C

?

文

a sketch of a hanging lantern at night

a sketch of a winding road through a desert

a sketch of a penguin on an ice floe

a sketch of a key lying on a wooden table

a sketch of a waterfall cascading down rocks

a sketch of a pair of scissors cutting paper

a sketch of a street lamp in light fog

a sketch of a cactus in a clay pot

a sketch of a violin leaning on a chair

a sketch of a squirrel holding an acorn

a sketch of a single leaf floating on water

a sketch of an open umbrella against wind

a sketch of a stone arch bridge at dawn

a sketch of a chess piece on a board

a sketch of a crow perched on a fence

a sketch of a winding staircase in a tower

a sketch of a person doing yoga pose

a sketch of a winding river through hills

L Text Prompts for Fig. 5

The textual prompts for the top-right of Fig. 5 in each row:

a sketch of an apple

a sketch of a teddy bear

a sketch of iron man

a sketch of Albert Einstein

Top-left:

a sketch of Eiffel Tower

a sketch of a dog wearing a hat

a sketch of a birthday cake

a sketch of a coconut tree

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: We have provided the details of the proposed module and extensive experiment results to support our claim.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We discuss that the balance between style consistency and content alignment is a tradeoff problem in Quantitative Analysis. In the Conclusions section, we point out the features that our approach currently leaves unimplemented. We also include a Limitation section in the Appendix.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: We propose a novel framework for sketch generation, but not a general theory. Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We have provided the implementation details in the main text. The code is in the supplementary materials, and more details are in the Appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We attach the code in the Supplementary materials. The dataset 4skst used in our paper is public, and we provide the selected data of style 5 in the Appendix.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: In the Section Experiments, we provide the implementation details, and more information is in the Appendix.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: The commonly used CLIP-T score and DINO score are used to measure the image-text alignment and style consistency.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: The experiments compute resources of our method are included in the last sentence of Quantitative Analysis.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We meet the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: In the qualitative analysis, we explained why our method can assist artist creation. We also include a social imapact section in the Appendix.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: The models compared in this paper are open-source.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [NA]

Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a text-to-sketch generation method. The method use reference features as auxiliary information with linear smoothing and use a style-content guidance mechanism. The framework can be extended to handle multiple styles (interpolating between different styles). Experimental results show that the proposed method can provide better style-content disentanglement.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n+ The experimental results shows that the proposed text-to-sketch method outperforms existing baseline approaches, both quantitatively and qualitatively.\n+ The explanation of the proposed method is clear and the paper is easy to follow.\n\nWeaknesses:\n- The main weakness is that text-to-sketch generation might be a sub-domain of text-to-image generation. It is questionable whether it is necessary to design a sophisticated method for the text-to-sketch task. The author mentioned in the paper that existing text-to-image models are not good at sketch generation, e.g., they are designed for color and natural images. However, this problem might just come from training data. If we replace training data, or finetune the existing model with additional sketch images, or train a sketch lora, this problem might have been solved. Different trigger words can be added to training captions to incorporate different sketch styles. \n\n- The proposed method uses null-text-inversion. However, this method usually requires optimization and make the method slow. The paper mentions \"Each sketch takes about 40 seconds (M3S (SD v1.5)) and 70 seconds (M3S (SDXL)) on an A100 40GB GPU\", but there is not comparison with baseline approaches about generation time. The proposed method might be significantly slower than existing approaches, limiting the applications of this approach.\n\nQUESTIONS:\n- As mentioned in \"Strengths And Weaknesses\", the generation speed might be an issue. It would be good to list the break down of time usage for different modules so that the readers can better understand the bottleneck of the proposed method.\n\n- The paper compares with many general baseline works that are designed for general styles. It would be good if the author can clearly mark which methods are designed for general styles, while which methods are designed for sketches only. This will help readers understand the scope of the baseline approaches.\n\n- It is usually subjective to evaluate the quality of a sketch. It would be more convincing if user study results are included in the paper.\n\nLIMITATIONS:\nThe authors have discussed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a method called M3S for generating stylized sketches from a text prompt, guided by one or two reference sketches. Instead of training a new model, the authors use a pre-trained Stable Diffusion model and inject style features by modifying the key (K) and value (V) components in the attention layers of the pre-trained model. They also leverage an AdaIN-based control mechanism that enables users to interpolate between different sketch styles. They propose a dual guidance formulation that separates text (content) and style impact, allowing for more fine-grained control over the generated sketches. Quantitative results suggest partial improvement over existing methods. This model also supports multi-style blending, which can be the unique feature of this model.\n\nSTRENGTHS AND WEAKNESSES:\nThanks for the authors effort to prepare the manuscript to advance the reference-based sketch extraction domain. Here, I listed the strengths and room for improvement:\n\n**- Strengths:**\n\n**1. [Simple yet effective model]:** I like the simplicity of the proposed method. It does not require re-training and it is built upon the existing knowledge in feature fusion and blending. It uses existing methods (e.g., K/V blending, AdaIn, and dual guidance) to merge style and content features in a smart way. \n\n**2. [Simple control over generated sketch for a human user]:** From a user point of view, the method seems to give lots of freedom in the generated sketch. Users can easily define two reference styles and control the generated sketch to be similar to one of those styles or to even have in between styles. \n\n---------------\n\n**- Room for improvement:**\n\n**1. [Lack of a user study]:** Sketches are sparse and sometimes quantitative metrics might not be the best representative of human opinion and evaluation. I believe a user study might be needed to show how much this method outperforms existing baselines (especially considering that in some quantitative metrics, this method seems to not be the best performing one).\n\n**2. [Limited technical novelty]:** The proposed method builds on many existing ideas in the literature, which could be seen as a limited technical novelty. However, if the model clearly outperforms previous baselines, I would consider this a minor issue. For example, AdaIN blending has long been used for arbitrary style transfer. K/V injection (substitution) in diffusion attention layers has been done before and here the feature fusion is improved to avoid artifacts.\n\n**3. [Quantitative evaluation of multi-style interpolation]:** While style blending is demonstrated visually, there are no metrics provided to quantify the quality of the style interpolations; such as the smoothness of transitions or the similarity of interpolated sketches to either reference (ref1 or ref2). It is also important to evaluate the consistency of interpolation quality across different pairs of reference styles, especially when the styles differ significantly from each other.\n\nQUESTIONS:\nTo strengthen the paper, I recommend the following improvements:\n\n**1. [Conducting a user study]:** Due to the subjectivity of sketch aesthetics, conducting a user study could help to verify this method superiority to existing baselines.\n\n**2. [Handling diverse reference styles]:** I would like to see how this model handles interpolating between styles, especially when two styles are really different from each other.\n\n**3. [Eq. 5 might be better represented]:** I think authors might improve the representation of Eq. 5. After reading the text, I could figure it out but it should somehow be more clear in the equation (somehow it should show where blended K/V features are used). \n\n**4. [Failure cases]:** It will be great if authors mention which kind of styles result in failure of this model. For example, can this model imitate the style of pointillism sketches (sketches drawn by many points)?\n\n**5. [Few missing references]:** Some reference-based sketch extraction studies from the computer graphic domain might be missing (see below). This is not a main concern; and I trust the authors to add if they found new ones; if not, it was a very minor comment that they can skip it. Here I added examples of missing studies that introduced style transfer concept into the sketch domain: [1] SketchPatch: Sketch stylization via seamless patch-level synthesis. TOG 2020, [2] Reference Based Sketch Extraction via Attention Mechanism, TOG 2022.\n\nI am on the positive side for this paper. If authors clearly demonstrate that their method is outperforming existing baselines or addressing some of the main concerns (like human evaluation), I am willing to increase the score after reading the author comments.\n\nLIMITATIONS:\nIt can be improved by showing failure cases, as it was explained in the review.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper presents a novel training - free diffusion model framework for multiple - style references. By integrating reference features as auxiliary information through linear smoothing, it effectively suppresses reference - sketch content leakage. Meanwhile, via a joint AdaIN module for coordinating multi - style generation, it achieves flexible style control without large - scale training, enhancing synthesis quality and expanding diversity.

STRENGTHS AND WEAKNESSES:
This paper presents a novel training - free diffusion - model - based framework. It can combine textual prompts and reference - style sketches to achieve accurate style guidance. Experiments show its significant advantages in style - alignment accuracy and control flexibility. The paper is well - structured, with clear method descriptions and detailed arguments. However, the need to adjust many hyperparameters restricts its practical - application convenience.

QUESTIONS:
This method has certain bias when processing certain objects. As shown in the clown example in Figure 1, even with $\eta$ set to 0, the generated clown image doesn't have a simple drawing style, though clowns are suitable for simple drawing. This echoes the fact that the method doesn't achieve optimal results in all cases in Table 2. What's the deep reason for this?

The paper only shows results of referring to multiple styles, but the used reference sketches are all of high - quality. So when encountering cases with big differences in style and drawing quality, the method's performance is unclear. For example, what's the effect when using amateur sketches (such as those in QuickDraw) and professional sketches as style references?

LIMITATIONS:
Referring to the suggestions put forward in the Questions
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a training-free framework for style transfer for sketch generation (image of a sketch, rather than a vector), including the ability to interpolate or use multiple reference styles. The framework is applicable for pretrained diffusion models, and the results are shown on Stable Diffusion / SDXL.\n\nThe core of the proposed technique is to, instead of using the K and V from the style source in the attention mechanism during the generation of the target image, to concatenate the style K and V matrices to the target t ones (potentially after linearly blending them with the target ones. For blending multiple styles, the linear combination of the latent images is used during the generation process.\n\nAuthors perform evaluation based on the style source images in Sketchy and 4skst. The textual prompts are generated using DeepSeek. CLIP-T score is used to measure alignment between prompt and generated images, and the distance in the space of DINO/VGG features is used to measure the style alignment with the reference style image.\n\nSTRENGTHS AND WEAKNESSES:\nQuality & Clarity: Authors clearly motivate their approach and discuss techniques that didn't work (ex. the use of AdaIN from Style aligned paper in lines 139-140). They perform the ablation study on the proposed modifications and do extensive comparison with state-of-the-art, outperforming it in the majority of cases. The writing is clear and there are plenty of examples in the paper. Supplementary material contains the code necessary for reproducibility for both SDXL and Stable Diffusion 1.5, as well as the source style images.\n\nSignificance & Originality: The proposed technique is novel and well-motivated for the application domain (which is somewhat narrow, as some of the assumptions made by the authors rely, for example, on images being grayscale).\n\nQUESTIONS:\nWhile the general motivation for introducing the content and style guidance losses (eq 5 is clear), the motivation to linearly increase the weight of the style guidance through the generation process is not clear. Why should this be done and how sensitive is the method to the schedule?\n\nFor the SDXL Implementation, the authors mention that style injection is important for the last layers, but injection into all layers compromises text alignment. Is there a principled way for deciding which layers to use, or some motivating ablations around it?\n\nLIMITATIONS:\nYes - the paper addresses many potential issues, and acknowledges a limitation when dealing with extremely sparse reference sketches where content is concentrated in a small area.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "CLARITY": 4, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-04"}