DiffInk: Glyph- and Style-Aware Latent Diffusion Transformer for Text to Online Handwriting Generation
Authors: Wei Pan
Huiguo He
Hiuyi Cheng
Yilin Shi
Lianwen Jin
Venue: ICLR 2026 Poster (2026)
Keywords: Text-Line Generation
Online Handwriting
Latent Diffusion Transformer
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We propose DiffInk, the the first latent diffusion Transformer framework for text to online handwriting generation (TOHG).

Abstract: Deep generative models have advanced text-to-online handwriting generation (TOHG), which aims to synthesize realistic pen trajectories conditioned on textual input and style references. However, most existing methods still primarily focus on character- or word-level generation, resulting in inefficiency and a lack of holistic structural modeling when applied to full text lines. To address these issues, we propose DiffInk, the first latent diffusion Transformer framework for full-line handwriting generation. We first introduce InkVAE, a novel sequential variational autoencoder enhanced with two complementary latent-space regularization losses: (1) an OCR-based loss enforcing glyph-level accuracy, and (2) a style-classification loss preserving writing style. This dual regularization yields a semantically structured latent space where character content and writer styles are effectively disentangled. We then introduce InkDiT, a novel latent diffusion Transformer that integrates target text and reference styles to generate coherent pen trajectories. Experimental results demonstrate that DiffInk outperforms existing state-of-the-art (SOTA) methods in both glyph accuracy and style fidelity, while significantly improving generation efficiency.

Published as a conference paper at ICLR 2026
DIFFINK: GLYPH- AND STYLE-AWARE LATENT DIFFUSION TRANSFORMER FOR TEXT TO ONLINE HANDWRITING GENERATION
Wei Pan, Huiguo He, Hiuyi Cheng, Yilin Shi,
Lianwen Jin"
South China University of Technology eewpan@mail scut. edu. en, fhehuiguo, eelwjin)@scutt.cn edu cn
ABSTRACT
Deep generative models have advanced text-to-online handwriting generation (TOHG), which aims to synthesize realistic pen trajectories conditioned on tex- tual input and style references. However, most existing methods still primarily focus on character- or word-level generation, resulting in inefficiency and a lack of holistic structural modeling when applied to full text lines. To address these issues, we propose DiffInk, the first latent diffusion Transformer framework for full-line handwriting generation. We first introduce Ink, a novel sequential variational autoencoder enhanced with two complementary latent-space regular- ization losses: (1) an OCR-based loss enforcing glyph-level accuracy. and (2) a style-classffication loss preserving writing style. This dual regularization yields a semantically structured latent space where character content and writer styles are effectively disentangled. We then introduce InkDiT, novel latent diffusion Transformer that integrates target text and reference styles to generate coherent pen trajectories. Experimental results demonstrate that DiffInk outperforms ex- isting state-of the-art (SOTA) methods both glyph accuracy and style fidelity, while significantly improving generation efficiency.
Full-line Text Input:
Tilhi 7 T-H
Style Reference
T
Generated Online Handwriring
S₁
G
3
4/ V/x
2
$ 1/ 7
S2
DiffInk (Ours)
G2
224 43
S3
3 2 14 28
-. 4
2 36
Figure 1: Overview of DiffInk. By directly modeling entire text lines rather than individual char- acters, the model efficiently synthesizes online handwritten text lines (G,) conditioned on textual input (T) and style references (S)), achieving accurate content reproduction and consistent style in both character form and layout structure. Different colors represent distinct handwriting styles.
INTRODUCTION
As shown in Figure 1, TOHG refers to the task of synthesizing realistic pen trajectories conditioned on textual content and style references. It has wide range of applications, including personalized digital ink rendering, handwriting simulation, and data augmentation for optical character recog- nition (OCR). However, TOHG presents unique challenges: beyond character-leved generation, requires coherent inter-character modeling and line-level layout; Compared to image-based meth- ods, it needs to handle temporal dynamics while maintaining glyph fidelity and stylistic consistency.
In recent years, TOHG has attracted growing research interest. Early methods (Graves, 2013; Li et al., 2024) adopt Auto-Regressee (AR) modeling to generate trajectory points sequentially. Sub- sequent approaches attempt control style through content-style disentanglement, utilizing either 1) coarse-grained global style representations such as writer IDs (Luo et al., 2022) 2) fine-grained local style representations such as character components (Liu al., 2024; Dai et al., 2023). Notably,
"Corresponding authors. Code available hutps//giihuukccm///wed///////

---

Published as a conference paper at ICLR 2026
Vanilla VAE
VA (Ours)
Vanilla VAE
InkVAE (Ours)
(a) Writer-style +SNE visualization
(b) Character-Ievel I-NE visualization
Figure 2: Latent-space visualization of Vanilla VAE vs. InkVAE (ours). While both mod- els achieve good reconstruction, Ink learns more structured latent space (visualized with t- SNE (Maaten & Hinton, 2008)): (a) Text-line features from
exhibits clearer writer-specific clusters. (b) Character-level features from 8 common characters-Ink' yields tighter intra-class groupings and more distinct inter-class separation.
OLHWG (Ren et al., 2025) adopts a novel diffusion- peafus framework to generate isolated characters and arranges them using external layout prediction modules, achieving SOTA performance. How- ever, these methods all fundamentally operate at the character or word level, lacking the ability to capture long-range dependencies and overall lext-line structure.
To tackle these limitations, we propose DiffInk, novel glyph- and style-aware conditional la- tent diffusion Transformer for TOHG. DiffInk begins by training a sequential variational autoen- coder (VAE) to learn compact latent representations of full text lines, thereby avoiding character-by- character modeling and capturing long-range structural dependencies. However, although a vanilla VAE can accurately reconstruct handwriting sequences when trained with only reconstruction loss, adding auxiliary perceptual losses directly on the generated outputs yields limited benefit, as the representations it learns lack semantic structure. Specifically, the VAE fails to group features by writer identity or character label in its internal representation space. As illustrated in the left panels of Figure 2 (a) and (b), features from different writers or characters overlap in an unstructured man- ner. As result, small perturbations introduced by the diffusion model in this space can easily push the generated output toward an incorrect writing style or an unintended character.
To resolve this, we introduce two lightweight regularization losses to regulate semantic structure in the latent space: (1) an OCR-based loss ensure glyph-level accuracy and (2) a style-classfficttion loss to enforce writer-level consistency. These losses are implemented by two trainable classifiers directly applied the latent space and jointly optimized with the VAE reconstruction loss. We refer to the resulting enhanced VAE as InkVAE, which learns well-structured latent space where glyph and style features are both effectively disentangled, enabling precise control over content and style generation. As evidenced in the right panels of Figure 2 (a) and (b), this regularization substantially restructures the latent space, yielding distinct clustering by both character and writing style. This regularized latent space offers robust foundation for conditional generation.
Building on this regularized latent space, we then develop novel InkDiT that synthesizes full-line handwriting conditioned on both target text sequences and short reference trajectories. The InkDiT denoises latent representations concatenated with textual content and style features, predicting clean outputs through iterative refinement. Specifically, textual content features are obtained by sequen- tially mapping characters to embeddings from a learnable codebook, followed by a lightweight content encoder. Style features are extracted from the reference trajectories using our Ink VAE coder. These features encompass both local attributes (c.g., character shapes and strokes) and global attributes (e.g., spacing and alignment). Subsequently. InkDiT effectively leverages these conditions to generate content-accurate and style-consistent results.
Our main contributions can be summarized as follows: (1) We propose DiffInk, the first latent diffusion framework for end-to-end full-line handwriting generation, capable of generating glyph- accurate and style-consistent pen trajectories.
(2) We introduce Ink VAK, sequential VAE enhanced with two task-relevant lightweight regularization losses (OCR-based losy and style-classffication loss) to disentangle content and style in the latent space, enabling structured representation learning. (3) We propose a novel Ink that jointly conditions on target text and reference style, generating realistic handwriting trajectories via iterative denoising process. (4) Experiments the CASIA- OLHWDB 2.0-2.2 benchmark demonstrate that DiffInk significantly outperforms current SOTA methods in full-line generation quality, while also offering improved efficiency.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
2.1 ONLINE HANDWRITING GENERATION
Early methods for online handwriting generation were predominantly RNN-based. Approaches such Graves' seminal work (Graves, 2013) and DeepWriting (Aksan et al., 2018) adopted L.TTMs with mixture density networks to model temporal dynamics, but lacked explicit style control. SketchRNN (Ha& & Eck, 2017) used an RNN-based VAE to generate ink data unconditionally. More recently, Trink (Jin et al., 2025) introduced a Transformer-basee architecture, which leverages global attention to generate more realistic and coherent handwriting trajectories.
In contrast to Latin- based scripts, languages like Chinese possess a vast set of characters with highly complex structures and stroke compositions, which has led the proposal of specialized approaches to capture these intricacies. FontRNN (Tang et al., 2019) incorporated monotonic atten- tion for stroke-level synthesis, while CHWmmateer (Li et al., 2024) utilized a sliding-window RNN for few-shot personalized generation. SDT (Dai et al., 2023) employed dual-branch encoders with contrastive learning to disentangle writer style from character content, and WLU (Tang & Lian, 2021) applied meta-lcarning to support rapid personalization. Building on the diffusion paradigm, DiffWriter (Ren et al., 2023) formulated character- synthesis as conditional denoising process, and OLHWG (Ren et al., 2025) further extended this framework by decoupling glyph synthesis and layout planning, thereby enabling layout-aware generation of full handwritten text lines. However, these approaches essentially remain focused modeling individual glyphs, with limited efforts toward directly modeling entire text lines. While OLHWG assumes that character generation is in- dependent of layout position and attempts to decouple text line into isolated characters and layout modules, such assumptions fail to capture the complexity of real handwriting. Appendix D.1 further discusses our distinctions from existing character-level and layout-decoupee approaches.
2.2 STYLE TRANSFER FOR HANDWRITING GENERATION
Controlling both content and style remains central challenge in handwriting generation. For con- tent representation, most methods employ content encoders to extract structural information from template inputs (Pippi et al., 2023). Some approaches (Pan et al., 2023; Kang et al., 2021) further design fine-gras tne-grained encoders to capture detailed local patterns. Alternatively, several methods (Ren et al., 2023; 2025) adopt learnable character embeddings to provide high-level semantic information. For style representation, early approaches (Kang et al., 2020; Gui et al., 2023) employed learnable style embeddings to represent the writing style of specific authors. Later methods (Dai et al., 2023; Liu et al., 2024) introduced style encoders to extract style features from few reference samples, using mechanisms such as cross-attention or adaptive normalization (Huang & Belongie, 2017) to guide the generation process and maintain stylistic consistency. (Kotani et al., 2020) obtains style representations via matrix factorization. For text-line handwriting generation, using continuous text-line reference (Li al., 2024; Pippi al., 2025) has become a popular approach to encode local writing style and overall layout.
2.3 DIFFUSION GENERATIVE MODELS
Diffusion models (Ho et al., 2020) have emerged as powerful tools in generative modeling, demon- strating impressive performance across images (Rombach et al., 2022), speech (Lee et al., 2025), and
tasks (Luo et al., 2023). To reduce computational costs and enhance semantic control, many recent works (Zhang et al., 2023; Saharia et al., 2022; Ramesh et al., 2022; Labs, 2024) adopt latent diffusion models, where the denoising process is conducted in a learned latent space. Recently, Diffusion Transformers (DiT) (Peebles & Xie, 2023) have replaced traditional U-Net (Ronneberger et al., 2015) backbones with Transformer-based (Vaswani al., 2017) archi- tectures, enabling stronger global conditioning in image synthesis tasks. Variational Autoencoders (VAEs) (van den Oord al., 2017: Baevski et al., 2020) are commonly used construct such spaces, offering compact and structured representations.
Overall, while considerable progress has been made in handwriting generation and latent diffusion modeling, generating complete online handwritten text lines remains a challenging and underex- plored task due to long-range dependencies and complex layouts. To this end, we propose DiffInk, structure-awasd generative framework based on a latent diffusion Transformer, providing a promis- ing direction for tackling text-to-online handwriting generation.
3

---

Published as a conference paper at ICLR 2026
Timestep 1
DiT Deneiser
Nossed X
Regullrtzztiinn brunch
Concal
Style: Teat
Classiter
Model
Gnooder
Nakio NAko mhottng
Getty
VAE Encoder
8
VAE Encoder
VAE Encode!
Witte
Text:
RABHA. HARSLLI
Lookee
fre Caleuee
2 t #2
Nyst
X
(a) InkVAE Training
(b) InkDiT Training
Figure 3: Overview of the DiffInk Framework. (a) InkVAE encodes online handwriting sequences into compact latent representations. During training, regularization losses Loer and Lat are applied to the latent space to encourage disentangled glyph and style. (b) InkDiT leverages this latent space to synthesize handwriting by denoising noisy inputs Dy into clean representations ro- The process conditioned content features Z obtained from text embeddings and style features Jzet derived from a reference trajectory. InkDiT is trained with a diffusion loss Lair-
METHOD
3.1 PRELIMINARY
Online Handwriting Data An online handwriting text-line data is represented as a pair (T, X), where T = (4,,12....,,mm denotes a sequence of m characters, and X {seqq seq.:..., seqm is the corresponding sequence of pen trajectories. Each seq; RT x5 encodes the trajectory of character t; with n; points. The total number of points for the entire line is Σ n, Each point is represented as 5-dimensional vector (æ., y, pen., where (I, y) are spatial coordinates and pen is one-hot vector indicating the pen state: Pen Down, Pen Up. or End of Character.
Task Overview Compared with single-character handwriting generation, TOHG is considerably more complex and challenging. First, text-line generation often involves much longer sequences (typically over 25 characters), which substantially increases modeling difficulty. Moreover, addi- tion to capturing handwriting style, TOHG also involves modeling inter-character dependencies and structural relationships across the line to produce fluent and natural handwriting. To make the prob- lem precise, we therefore formalize TOHG as follows: the input consists of reference trajectory Xief and a concatenated text combining Teer and T gens and the goal is to generate trajectory Xgen that renders T gen while preserving the handwriting style in X
3.2 OVERVIEW OF THE DIFFINK FRAMEWORK
As illustrated in Figure 3, DiffInk consists of two key components: Ink VAE, a pre-trained sequential variational autoencoder, and Ink conditional latent diffusion Transformer. Ink compresses online trajectories X E IR- into latent representations E Rixd, capturing both glyph-level structures and writer-specific style features. InkDiT operates within this structured latent space. receives noisy latent imput x; along with two conditioning signals: (1) a style prompt Tee E gixd obtained by encoding the reference trajectory X with the Vnk encoder; and (2) a content condition Z R encoded by the content encoder from text embeddings. Guided by these conditions, InkDiT denoises Ię to reconstruct the clean latent representation To. During inference, the process starts from pure Gaussian noise and, under the same content and style conditions in training, applies DDIM sampling to obtain a clean latent representation. This refined latent is then decoded by the Ink decoder to generate the final handwriting sequence.
3.3 GLYPH- AND STYLE-AWARE INKVAE
As shown in Figure 3 (a), Ink employs an mecoer-ddcoode architecture, primarily designed for efficient modeling of handwriting sequences through a compact latent space. Beyond this, VAE incorporates novel task-relevant regularization strategy construct well-structured latent space,
4

---

Published as a conference paper at ICLR 2026
which promotes the disentanglement of content and style, leading to robust representations across both diverse handwriting styles and character classes. Such structured representations, as evidenced by recent studies (Tang et al., 2025; Yao et al., 2025; Guo al., 2025), have been shown to improve the convergence efficiency and overall performance of downstream diffusion models.
Trajectory Encoder To extract complex handwriting features, we employ ID convolutional en- coder to transform the input handwriting sequence X E R into a compact latent representation E R This representation is directly provided input to three branches: (1) a trajectory decoder for handwriting reconstruction, (2) a Transformer-based OCR module for text-line hand- writing recognition, and (3) style classifier for writer identification.
Trajectory Decoder Following (Tang et al., 2019), the decoder first applies a 1D convolutional stack that mirrors the trajectory encoder, up-sampling the latent feature Φ into feature map O₁ R. (2Nx()))3) The feature vector O is divided into two branches: (1) 6p logits that parameterize pr component Gaussian Mixture Model (GMM) (Reynolds, 2015) for predicting the (z, y) coordinates, and (2) 3 logits for classifying the pen state pen. Accordingly, the reconstruction loss is defined as free = fren + Lgmm- where fipe is a focal loss used for the three-way pen-state classification, and f.gmm is the negative log- ligeliielihood of the ground-truth points under the predicted mixture. To enable the model to learn when to stop writing, the GMM loss is computed only over valid trajectory steps, while the pen-state loss applied across the entire sequence, including padded regions.
Glyph- and Style-Aware Regularization To encourage the latent space to preserve character- level structural information, introduce lightweight OCR module structural regularizer. Operating at the feature-token level, this module promotes consistency in the representations of the same character class across different writing styles. Specifically, the OCR module decodes latent representations into character sequences using Transformer-bass recognition head, and is trained with a CTC-based (Graves al., 2006) loss, denoted as for- This supervision imposes structural constraints on the encoder, promoting the learning of pharacter-invarian features and enhancing both the structural awareness and semantic disentanglement of the latent space.
On the other hand, to enhance the discriminability of writing styles in the latent space, we introduce lightweight style encoder latent representation to extract a holistic style representation. It summarizes the latent features using an LSSTM network (Staudemeyer & Morris, 2019) with attention pooling and is optimized by writer classification loss (Cuy), which explicitly encourages style-aware separation in the latent space.
global discriminative supervisor. This module processes the entire
VAE Loss Function Ink is trained end-to-end with the objective function LVA defined in Equation 1. This training strategy enables the encoder to learn latent representations that are struc- turally coherent, semantically aligned, and stylistically informative, providing solid foundation for diffusion- based handwriting synthesis. Details of Ink VAE are presented in the Appendix B.1.
LVVE = Σ De: Le where & frec, kl, ocr, sty}
3.4 HANDWRITING GENERATION WITH INKDIT
As shown in Figure 3 (b), InkDiT employs a Transformrr diffusion architecture for con- ditional online handwriting generation in the latent space. It builds on InkVAE's compact and structured representation and progressively refines noisy latent variables into coherent trajectories through denoising. By conditioning on both text-line Tie + T and a reference trajectory X InkDiT achieves controllable generation with improved fidelity and continuity.
For content representation, each character in the input text Tre + Tgen is em- bedded via learnable codebook E R x where K is the vocabulary size and diea is the em- bedding dimension. By embedding characters directly, the approach learns high-level semantic representations of characters, while avoiding the domain gap caused by the complex mapping from fixed templates to long handwritten text-line sequences (Wang et al., 2023; 2026). Text-line gener- ation is further challenged by uncertain character lengths at inference, which prevent direct embed- ding- traje-ctryytryyyoooo alignment as in character-level methods (Ren et al., 2023; 2025). We address this through simple yet efficient design where character embeddings are sequentially concatenated,
Text-Line Content
5

---

Rixde serving as Published as a conference paper at ICLR 2026
padded with shared learnable embeddings to match 2u, and processed by lightweight content coder. Based on ConvNeXt-V V2 (Woo et al., 2023), this encoder leverages large-kerne depthwise convolutions to capture long-range dependencies and yields representation Z semantically aligned content conditions for generation guidance.
Text-Line Handwriting Style For style representation, character-level approaches (Dai et al,, 2023; Tang & Lian, 2021; Ren et al., 2025) typically rely on randomly sampled isolated characters from the same writer. contrast, text-line generation requires capturing additional layout informa- tion absent in such isolated samples. To holistically model both writing style and text-line layout, we follow (Li et al., 2024; Pippi et al., 2025) and adopt continuous handwritten trajectory Nee as the style reference. This trajectory is encoded into latent representation by the trajectory encoder of our proposed style-aware Ink VAE. Leveraging the VAE encoder's ' strength as a versatile feature extractor (Leng et al., 2025; Cheng & Yuan, 2025), obtain the style feature zret- By modeling style directly within the unified latent space of the VAE (Tan et al., 2025; Chen et al., 2025), we avoid introducing separate style encoder and ensure consistent representation learning.
DiT Denoiser The diffusion network follows the DiT (Peebles & Xie, 2023) architecture, leverag- ing its contextual modeling capacity to recover the clean latent from noisy inputs under the guidance of content features Z and style features Fref. The three inputs are concatenated along the channel dimension and linearly projected to align with the VAE feature space.
Zt
Vās- + vI Or (~,((0,)
(2)
20 InkDiT, a ([Te; Jeef 21, t);
Lait Ex,, [mask @
|&g xo 21
DiT denoiser is trained to recover the clean feature zo at masked positions by minimizing the masked mean squared error (MSE), as defined Equation 3, where the masked regions correspond to the reference style trajectory features. Benefiting from the semantically structured latent space provided by Ink the DiT model can more effectively utilize conditional inputs to recover features from noise, resulting in better alignment with the underlying data distribution. Following the DDIM (Song et al., 2020) sampling strategy, we start from Gaussian noise and iteratively generate the final latent estimate ân. conditioned on both content and style features. Finally, this latent representation is de- coded by the Ink VAE decoder to reconstruct the handwriting trajectory. Details InT,, including its architectural design well training and inference procedures, presented in Appendix B.2.
EXPERIMENTS
4.1 DATASET & EVALUATION METRICS
Dataset We conduct experiments to validate DiffInk on Chinese handwriting datasets. For Chi- nese TONG, use CASIA-OLHWDB 2.0-2.2 (Liu et al., 2011), sampling 500 writers based on longer average text lines. Among them, 400 writers are used for training and 100 for testing. We then augment the training set with synthesized samples, yielding 67,000 text lines covering 2,648 characters. The test set comprises 4,780 text lines. Details are presented in the Appendix A.
Evaluation Metrics To comprehensively evaluate the proposed DiffInk, following prior works (Tang & Lian, 2021; Dai et al., 2023; Ren et al., 2025), we employ several widely used evaluation metrics, including: (1) Content fidelity, assessed with pretrained text-line OCR model by reporting Accurate Rate (AR) and Correct Rate (CR) following the (Yin et al., 2013) OLHCTR benchmark; (2) Style consistency, measured with writer classifier via writer classification accuracy (Style); (3) Trajectory similarity, evaluated using normalized Dynamic Time Warping (DTW) dis- tance (Berndt & Clifford, 1994; Chen et al., 2022); (4) Generation efficiency, reported as the number of characters generated per second (Avg. char/s). Details are presented in Appendix B.4.
4.2 IMPLEMENTATION DETAILS
For Chinese TOHG, Ink VAE is trained for 100 epochs with batch size of 128 and learning rate of 5 x 10-4, To balance different objectives, apply the following loss weights: Agmm 1.0.

---

Published as a conference paper at ICLR 2026
Table 1: Quantitative comparison with SOTA handwriting generation methods. Baseline meth- ods generate isolated characters and compose text lines through a shared layout prediction module from OLHWG. In contrast, our method directly generates complete text lines and outperforms these single-character approaches across all evaluation metries.
Method
Venue
Output
AR%1 T
CR% 1
Style
DTW
Avg. charls 1
Drawing (Zhang et al., 2017)
TPAMI'
Char-level
76.35
76.87
25.75
1.582
4.16
DeepImitator (Zhao al., 2020)
PR"20
Char-level
78.21
78.98
27.62
1.561
3,96
WLU (Tang & Lian, 2021)
CGF
Char-level
79.85
80.25
35.71
1.540
25.19
SDT (Dai et al., 2023)
CVPR'23
Char-level
82.53
83,00
50.51
1.270
3.35
OLUIGG (Ren al., 2025)
ICLR'25
Churtlaoout
91.48
91.71
44.74
1.326
0.07
Diffink
This work
Line-tevel
94.38
94.58
77.38
1.049
58,47
Ape = 2.0, Aver = 1.0, Xay = 0.5, and Au = 1 x 10-6.
The DiT model is trained for 200k steps
with a batch size of 256 and a learning rate of 7.5 x 10 Details are presented in the Appendix B.
4.3 MAIN RESULTS
Comparing with Representative Methods As there lack of established baselines for end- to-end text-line generation, we adopt several SOTA character generation methods and extend them with a layout modeling module to enable line-level comparison. These baselines include Draw- ing (Zhang et al., 2017), DeepImitator (Zhao et al., 2020), WLJ (Tang & Lian, 2021), and SDT (Dai et al., 2023), which generate high-fidelite handwritten characters under an autoregressive paradigm, as well OLHWG (Ren et al., 2025), which synthesizes isolated characters using a diffusion- based approach. To support text-line level comparison, we adopt the layout prediction module from LLHWG-currente
the SOTA-to compose full lines from the character-level outputs of all five methods. In the OLHWG method, the layout prediction module infers the subsequent layout based on the layouts of fixed number of preceding characters (originally set 10) from the ground-truth text line. To accommodate text lines of varying lengths, we instead use the layouts of the first 30% of characters from each text line as input for layout prediction (max 10). To ensure fairness, all methods are retrained on the same dataset with consistent preprocessing and default configurations.
Quantitative Evaluation As shown in Table 1, DiffInk (ours) achieves superior performance across all quantitative metrics. For content fidelity, the OCR- based metrics AR and CR both ex- ceed 94%, outperforming the latest SOTA method OLHWG by an average margin of 3 percentage points, and substantially surpassing earlier approaches such as Drawing, Deeplmitator, WLU and SDT-with margin of 14.44%. For style consistency. DiffInk achieves style score of 77.38 at the text-line level-the highest among all Suthhodss--pppsinin the contrastive-leaming--he HWG by 30 percentage points, the dual-branch style-loss-basee based SDT by 27 points, and baselines with simple style modeling by an average of 50 points. Moreover, style evaluation further reveals that layout stitching introduces anomalies that can be readily detected by the style classifier. In terms of structural fidelity, DiffInk attains substantially lower normalized DTW distances, indicat- ing more consistent alignment with target trajectories. In contrast, autoregressive methods such as WLU and SDT suffer from cumulative error propagation, which severely degrades text-line level performance. Finally, Difflnk demonstrates strong generation efficiency, producing 58.82 charac- ters per second-177 faster than SDT, over 800x faster than OLHWG, and more than 2x faster than WLU-thanks the efficient text-line modeling of InkE
OL-
Qualitative Evaluation As shown in Figure DiffInk (ours) produces more continuous and co- herent text lines with smooth character transitions. In contrast, character-wise methods such as SDT and OLHWG often exhibit unnatural stitching artifacts at character boundaries, while WLU tends to generate characters that stick together, likely due to its strong reliance on the pretrained con- sent-style encoder. These problems arise partly because character-level methods can only assemble text lines by scaling and positioning individual characters according to their predicted bounding boxes. During inference, obtaining these bounding boxes requires an additional prediction module, which introduces its own errors and further affects the reliability of the final text-line layout. Be- yond these practical limitations, although their single-character generation quality is generally good, the decoupled modeling of layout and characters neglects structural dependencies between adjacent characters. making it difficult to achieve consistent and coherent text-line layouts when character shapes differ substantially. Additional visualization results provided in Appendix C.
7

---

Published as a conference paper at ICLR 2026
Methed
Botoom- Left Top Right Lagrut
Honaoral Layour
Top Left Botton Right Layour
Brawing
Deep
#
Iniiax
WLU
* Kak $2.
*i 2. d
SDT
4.R (-d.::ll.00 b-
A 1 t
4-8. 4-4 6. 4.
#s 1E f kihss
$ 55
nimak
(Chrn)
# #:
of
1 & &oocat
26
Text
Brawing
tbas
Deep
1: and
Inivso
WL.
SD0
4 4 3
Ki 6...6..0..
OLEWG
fo 1 +. 000 22 sL.
4 - 22 Jn 82.
numtak
(Curs)
PG -i. N v dus 1
3 240 Al f
-6 8,de Hanniitl Aww.)) & v.
Figure 4: Comparison with SOTA methods under unseen writing styles across diverse layouts. All baseline methods generate isolated characters and compose lines via a shared layout module. Blue boxes denote the same style reference, while red boxes highlight errors or unnatural character connections. These methods suffer from stitching artifacts, especially when adjacent characters differ structurally. DiffInk generates more coherent and naturally connected text lines.
Table 2: Ablation study of Vnk and its impact on InkDiT generation. "V" means loss en- abled. While all VAEs achieve near-pertee near-perfect reconstruction performance with various losses, InkDiT achieves optimal performance when trained on Ink latents (last row). This confirms that our lightweight regularization losses structure the latent space for robust diffusion- genera- tion, with negligible impact VAE reconstruction performance.
VAE Losses
VAE Reconstruction Performance
InkDIT Generation Performance
Creesk
Leee
Lay
AR% 1
CR% 1
Style 1
DTW
AR%
CR% 1
Style t
DTW J
x
x
97.59
97.61
99.97
0.014
74.77
75.20
60.68
1.062
/
97.60
99.97
82.41
1.052
X
97.59
97.62
99.98
0.016
79.64
79.95
68.99
1.059
97.65
97.67
99.97
0.016
94.38
94.58
77.38
1.049
4.4 ABLATION STUDY OF INKVAR AND INKDIT
Effeet of InkVAE and impact on InkDiT As shown Table 2, we perform an ablation study to evaluate the effectiveness of InkVAR. The left columns list the loss configurations of different VAE variants, the middle columns report reconstruction performance, and the right columns present generation results of InkDiT trained on the corresponding feature spaces. Since online handwriting data is relatively clean and less affected by background noise than image-pixel modeling, VAEs trained solely with the reconstruction loss Lree already achieve high-fidelity reconstructions, flected by the low DTW distances in the middle columns. However, the diffusion results reveal a key limitation: strong reconstruction fidelity does not guarantee a well-structured semantically meaningful latent space. Compared with the Vanilla VAF, adding OCR regularization improves recognition accuracy by about 4 points, while style regularization yields an 8-point gain in the writer- classficaasiiiitiioonn score. When coabbined-formingg our proposed VAE- latent space be- comes markedly more suitable for diffusion modeling, as evidenced by the best overall results across recognition accuracy, style classification score, and DTW distance.
Figure 2 further visualizes the latent space. Unlike the Vanilla VAE, which fails to form clear clusters OCR and style regularizationnpprduucs well- separated clusters, yielding more discriminative and semantically meaningful representations. This shows that although the Vanilla VAE reconstructs sequences well, its latent features lack sufficient semantic information, so even small perturbations may lead to content errors or stylistic drift during
by writing style or character class,
8

---

Published as a conference paper at ICLR 2026
VAE Model
DIT generation results (trined on correspandingg VAE)
Table Ablation study of InkDiT. The con- tent encoder with ConvNextV2 significantly im- proves content accuracy and style consistency with minimal overhead, while introducing long- skip residual connections leads to performance degradation and additional computational cost.
Baseline
losR
W
M
+ Style losa
12.
-CCR Style
20 pq.
Reach
loos (uurs)
Target
Model
AR%
CR% Style 1 DTW Arg. char's
InkDiT Generation with VAE Vari- ants. Blue boxes highlight content errors; red boxes indicate style inconsistencies. InkDiT trained the latent space from our Ink yields more accurate and consistent results.
Figure 5:
w/ Longskip
71.73
71.91
55.42
1.074
55.55
wro ConvNextV2
81,51
81.71
65.49
1,052
71.94
DiffInk (Ours)
94.38
94.58
77.38
1.049
58.47
generation. Building on this, Figure presents qualitative results of InkDiT trained on different VAE variants. While the Vanilla VAE often produces inaccurate content and inconsistent styles, Ink achieves both high content fidelity and stylistic consistency, demonstrating the effectiveness of disentangled latent spaces for controllable handwriting generation.
As shown in Table 3, we per- form an ablation study on InkDiT. Introducing long skip connections results in degraded per- formance, whereas removing the content encoder causes a marked drop in content quality and fur- ther undermines style consistency. Interestingly, even with simple padding of text embeddings and without explicit sequence-length annotations, the content encoder effectively captures long-range dependencies and provides accurate semantic conditioning. This effectiveness can be attributed to its ConyNeXt-V2 V2 (Woo et al.. 2023) backbone with large convolutional kernels, which not only provide a broad receptive field but also integrate embeddings across spatial and sequential dimen- sions. Such integration yields more robust alignment of character content embeddings with the VAE latent space, preventing over-reliance on individual tokens. Moreover, by enriching the contextual information, the content encoder mitigates semantic ambiguity: without it, the model must rely on sparse text embeddings, which often leads to misalignment and confusion. The visual results in Fig- ure 6 further support these observations, demonstrating that, without the content encoder, generated text exhibits poor alignment and semantic drift.
Effect of InkDiT
Text
#
wio ConvNext
Xire
Dirlin(OUrs))
$ 1
3 2 A
Target
4x #
Ao A
Figure 6: Ablation study the content
coder. Without the content encoder, content er- rors (red boxes) occur frequently.
4.5 FINE-GRAINED CHARACTER- ANALYSIS
As shown in Figure 7, we visualize character centroids (red dots) for each method and connect them to reveal layout trends. Baseline methods exhibit unnatural connections between adjacent charac- ters in certain regions (highlighted by red boxes), stemming from their assumption that character content and layout can be modeled independently. This often results in broken baselines and dis- continuous structures. This phenomenon points to a fundamental property of handwritten text lines: character layout and connections cannot be fully disentangled, as they remain inherently context- dependent. Importantly, text-line layout is not merely arranging character bounding boxes; it should also account for the structural properties of individual characters and the contextual dependencies among their neighbors. Rather than treating layout as separate post-processing step, DiffInk di- rectly models the spatial arrangement and inter-character connections of the entire text line within a structure-aware latent space, thereby producing more coherent and consistent results.
Text
Drawing
DeopIminator
WLU
SDT
OLHWG
DiffInk (Ours)
Target
$ n 2
i8
#
Md
3R dut
Figure 7: Fine-Grained Layout Analysis. Bascline methods show local misalignments (red boxes), while DiffInk achieves more coherent and consistent layouts.

---

Published as a conference paper at ICLR 2026
-30
+ -4
Drawing
Deeplminator
WLU
SDT
OLHWG
DiffInk (Ours)
Figure 8: Comparison of data distributions between generated and real data. t-SNE (Maaten & Hinton, 2008) visualization of features from the same text lines generated by baselines and Difflnk. Difflnk closely overlaps with real data, while baselines show varying degrees of deviation.
4.6 VISUALIZATIONS WITH T-SNE
As shown in Figure 8, the t-SNE visualization of features extracted by the trajectory encoder reveals that Difflnk's generated text lines closely overlap with real handwritten samples, indicating that Difflnk effectively captures the distribution of authentic handwriting. In contrast, baseline methods that synthesize characters individually and concatenate them using bounding boxes exhibit a notice- able distributional shift from real data. This suggests that although text lines can be reconstructed through post-hoc concatenation, such pipelines struggle to model the coupling between character shapes and text-line layout, which in real handwriting arises from continuous writing dynamics and natural inter-character transitions. As result, the holistic structure of real text lines, whereas DiffInk's end-to-end modeling captures both global layout and stylistic coherence more faithfully. These results further confirm that modeling the entire text line as a unified sequence is essential for preserving realistic handwriting statistics.
methods fail to reproduce
CONCLUSION
In this paper, we propose DiffInk, a pioneering conditional latent diffusion Transformer frame- work for TOHG. Our key contributions are: (1) task-aware variational autoencoder (Ink that incorporates OCR-based and style regularization to learn a semantically structured latent space with disentangled content and style; and (2) conditional latent diffusion Transformer (InkDiT) that conditions on textual input and reference styles to generate coherent pen trajectories. Built upon these components, Difflnkk produces full-line handwritten text with high content accuracy, con- sistent style, and natural inter-characte transitions. Comprehensive experiments demonstrate that DiffInk significantly outperforms existing methods, validating the effectiveness of our approach. This work establishes the effectiveness of the conditional latent diffusion model for complex line handwriting generation and highlights its potential for OCR, personalized digital handwriting applications, and human-comppter interaction systems. Limitations and discussions, including ex- tensions to paragraph-level generation and to other languages are presented in Appendix D.
ACKNOWLEDGEMENT
This research is supported in part by the National Natural Science Foundation of China (Grant No.:62476093).
REPRODUCIBILITY STATEMENT
Reproduction consists of dataset preparation (Sections 4.1 and Appendix A), the design of DiffInk InnVAE and InkDiT) as described Sections 3.3 and 3.4, implementation and training detailed in Appendix B and Section 4.2, and evaluation following Section 4.3 and Appendix B.4.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Emre Aksan, Fabrizio Pece, and Otmar Hilliges, Deepwriting: Making digital ink editable via deep generative modeling. In Proceedings of the 2018 CHI conference on human factors in computing systems, pp. 1-14, 2018.
Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A frame- work for self-supervised learning of speech representations. Advances in neural information processing systems, 33: 122444142400 2020.
Donald Berndt and James Clifford. Using dynamic time warping find patterss in time series. In Proceedings of the 3rd international conference on knowledge discovery and data mining, pp- 359-370, 1994.
Bowen Chen, Mengyi Zhao, Haomiao Sun, Chen, Xu Wang, Kang Du, and Xinglong Wu. Xverse: Consistent multi-subject control of identity and semantic attributes via dit modulation. arXiv preprint arXiv:2506. 211.0006.1116, 2025.
Zhounan Chen, Daihui Yang, Jinglin Liang, Xinwu Liu, Yuyi Wang, Zhenghua Peng, and Shuang- ping Huang. Complex handwriting trajectory recovery: Evaluation metrics and algorithm. In Proceedings of the asian conference on computer vision, pp. 1060-1076, 2022.
Yu Cheng and Fajie Yuan. Leanvae: An ultra-efficient reconstruction vae for video diffusion models. arXiv preprini 20t::22033.33555 2025.
Gang Dai, Yifan Zhang, Qingfeng Wang, Qing Du, Zhuliang Yu, Zhuoman Liu, and Shuangping Huang. Disentangling writer and character styles for handwriting generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5977-5986, 2023.
DeepSeek-AL. Deepseek Ilm: Scaling open-source language models with longtermism. arXiv
preprint arXiv:2401. 200:22400.002544, 2024.
David H Douglas and Thomas K Peucker. Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Cartographica: the international journal for geographic information and geovisualization, 10(2):112-122, 1973.
Alex Graves. Generating sequences with recurrent neural networks.
arXiv preprint
1308. 0850, 2013.
Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist tem- poral classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning, pp. 369-376, 2006.
Dongnan Gui, Kai Chen, Haisong Ding, and Qiang Huo. Zero-shot generation of training data with denoising diffusion probabilistic model for handwritten chinese character recognition. In Document Analysis and Recognition ICDAR 2023, pp. 348-365 Springer, 2023.
Jinpei Guo, Zheng Chen, Wenbo Li, Yong Guo, and Yulun Zhang. Compression-awee aware one-step diffusion model for jpeg artifact removal. arXiv preprint arXiv:2502.088 2502.09873, 2025.
David Ha and Douglas Eck. A neural representation of sketch drawings. arXiv preprint arXiv: 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Xun Huang and Serge Belongie. Arbitrary style transfer in real real-time with adaptive instance normal- ization. In Proceedings of the IEEE international conference computer vision, pp. 1501-1510, 2017.
Zezhong Jin, Shubhang Desai, Xu Chen, Biyi Fang, Zhuoyi Huang, Zhe Li, Chong-Xin Gan, Xiao Tu, Man-Was Wai Mak, Yan Lu, and Shujie Liu. Trink: Ink generation with transformer network. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp- 4857-4866 2025.

---

Published as a conference paper at ICLR 2026
Lei Kang. Pau Riba, Yaxing Wang, Marçal Rusinol, Alicia Fornés, and Mauricio Villegas. Ganwrit- ing: content-conditioned generation of styled handwritten word images. In European conference computer vision, pp. 273-287222 229. Springer, 2020.
Lei Kang, Pau Riba, Marcal Rusinol, Alicia Fornes, and Mauricio Villegas. Content and style aware generation of text-line images for handwriting recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(12):8846-888) 8860, 2021.
Atsunobu Kotani, Stefanie Tellex, and James Tompkin. Generating handwriting via decoupled style descriptors. In European Conference on Computer Vision, pp. 764-780, Springer, 2020.
Black Forest Labs. Flux. https:/ /githtb......... www.cooo//cccooooooo forest- labs/aabsslls -orest-aabe/Eeus 2024.
Sang- Hoog Lee, Ha- Yeong Choi, Seung- -in Kim, and Scong- Whan Lee. Hierspecch++: Bridg- ing the gap between semantic and acoustic representation of speech by hierarchical variational inference for zero-shot speech synthesis. IEEE Transactions Neural Networks and Learning 2025.
Systems,
Xingjian Leng, Jaskirat Singh, Yunzhong Hou, Zhenchang Xing, Saining Xie, and Liang Zheng. Repa-e: Unlocking vae for end-to-end tuning with latent diffusion transformers. arXiv preprint arXiv:2504. 10483, 2025.
Shilong Li, Shusen Tang, and Zhouhui Lian. Chwmaster: mastering chinese handwriting via sliding- window recurrent neural networks. International Journal on Document Analysis and Recognition (LJDAR), 27(3)2275222 288, 2024.
Cheng-Lin Liu, Fei Yin, Da-Han Wang, and Qiu-Feng Wang. Casia online and offline chinese handwriting databases. In 2011 International Conference on Document Analysis and Recognition, pp. 37-41. JEEE, 2011.
Yu Liu, Fatimah Binti Khalid, Lei Wang, Youxi Zhang, and Cunrui Wang. Elegantly written: Dis- entangling writer and character styles for enhancing online chinese handwriting. In European Conference on Computer Vision, pp. 409 425. Springer, 2024.
Marcus Liwicki and Horst Bunke. lam-ondb-an on-line english sentence database acquired from handwritten text on a whiteboard. In Eighth International Conference Document Analysis and Recognition (ICDAR 005), pp. 956-961. IEEE, 2005.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv: 2017.
Canjie Luo, Yuanzhi Zhu, Lianwen Jin, Zhe Li, and Dezhi Peng. Slogan: handwriting style syn- thesis for arbitrary-Iength and vocabulayya text. IEEE transactions on neural networks and learning systems, 34(1 1):85)3-8511 2022.
Jianjie Luo, Yehao Li, Yingwei Pan, Ting Yao, Jianlin Feng, Hongyang Chao, and Tao Mei. Semantic-conditional diffusion networks for image captioning. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 23359-23368, 2023.
Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9NNov) 2008
OpenAl. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 2023.
Wei Pan, Anna Zhu, Xinyu Zhou, Brian Kenji Iwana, and Shilin Li. Few shot font generation via transferring similarity guided global style and quantization local style. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 19506-19516, 2023.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference computer vision, pp. 4195-4205, 2023.
Dezhi Peng, Canyu Xie, Hongliang Li, Lianwen Jin, Zecheng Xie, Kai Ding, Yichao Huang, and Yaqiang Wu. Towards fast, accurate and compact online handwritten chinese text recognition. In Document Analysis and Recognition ICDAR 2021, pp. 157-171. Springer, 2021.
12

---

Published as a conference paper at ICLR 2026
Vittorio Pippi, Silvia Cascianelli, and Rita Cucchiara. Handwritten text generation from visual archetypes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pp. 22458-224665 2023.
Vittorio Pippi. Fabio Quattrini, Silvia Cascianelli, Alessio Tonioni, and Rita Cucchiara. Zero-shot styled text image generation, but make it autoregressive. Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, 2025.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint 1xXi:22204001222 1(2):3, 2022.
Min-Si Ren, Yan-Ming Zhang, Qiu-Feng Wang, Fei Yin, and Cheng-Lin Liu. Diff-writer: diffusion model-based stylized online handwritten chinese character generator. In International Conference on Neural Information Processing, pp. 86-100. Springer, 2023.
Minsi Ren, Yan-Ming Zhang, and yi chen. Decoupling layout from glyph in online chinese hand- writing generation. In The Thirteenth International Conference on Learning Representations, 2025. URL https::////... /ttp:///pppeeiiieeeeeee nwt.
9Nb11.
Douglas Reynolds. Gaussian mixture models. In Encyclopedia of biometrics, pp. 827-832. Spriner,, 2015.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedi- cal image segmentation. In International Conference on Medical image computing and computer- Springer, 2015.
assisted intervention, pp.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Bureu Karagol Ayan, Tim Salimans, Jonathan Ho, David Fleet, and Mohammad Norouzi. Photorealistic text-to- cext-to-mmaee diffusion models with deep language understanding. Advances in neural information processing systems, 35:36479-36494, 2022.
Jiaming Song. Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502. 2020.
Ralf C Staudemeyer and Eric Rothstein Morris. Understanding Istm-a tutorial into long short-term memory recurrent neural networks. arXiv preprint
2019.
Zhenxiong Tan, Songhua Liu, Xingyi Yang. Qiaochu Xue, and Xinchao Wang. Ominicontrol: Mini- mal and universal control for diffusion transformer, In Proceedings of the IEEE/CVE International Conference Computer Vision, pp. 14940-14950, 2025.
Long Tang, Tingting Chai, Zheng Zhang, Miao Zhang, and Xiangqian Wu. Palmdiff: When palm- print generation meets controllable diffusion model. IEEE Transactions Image Processing, 34: 5228-5240, 2025.
Shusen Tang and Zhouhui Lian. Write like you: Synthesizing your cursive online chinese handwrit- ing via metric-hased meta learning. In Computer Graphics Forum, volume 40(2), pp. 141-151. Wiley Online Library, 2021.
Shusen Tang, Zeqing Xia, Zhouhui Lian, Yingmin Tang, and Jianguo Xiao. Fontrnn: Generating large-scale chinese fonts via recurrent neural network. In Computer Graphics Forum, volume 38(7), pp. 567-577. Wiley Online Library, 2019.
Aaron van den Oord, Oriol Vinyals, and koray kavukcuoglu. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural informa- fion processing systems, 30, 2017.
13

---

Published as a conference paper at ICLR 2026
Chi Wang, Min Zhou, Tiezheng Ge, Yuning Jiang, Hujun Bao, and Weiwei Xu. Cf-font: Content fusion for sho-shot font generation. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 1858-1867. 2023.
Honglie Wang, Minsi Ren, Yan-Ming Zhang, Fei Yin, and Cheng-Lin Liu, Template-guided cas- caded diffusion for stylized handwritten chinese text-line generation. In Document Analysis and Recognition ICDAR 2025. Springer, Cham, 2026.
Sanghyun Woo, Shoubhik Debnath, Ronghang Hu, Xinlei Chen, Zhuang Liu, So Kweon, and Saining Xie. Convnext v2: Co-designing and scaling convnets with masked autoencoders, In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16133- 16142, 2023.
Jingfeng Yao, Bin Yang, and Xinggang Wang. Reconstruction generation: Taming optimization dilemma in latent diffusion models. In Proceedings of the Computer Vision and Pattern Recogni- fion Conference, pp. 15703-15712, 2025.
Fei Yin, Qiu-Feng Wang, Xu- Yao Zhang, and Cheng-Lin Liu. Icdar 2013 chinese handwriting recog- nition competition. In 2013 12th international conference on document analysis and recognition, pp. 1464-1470. IEEE, 2013.
Lvmin Zhang. Anyi Rao, and Mancesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference computer vision, pp. 3836-3847, 2023.
Yao Zhang, Fei Yin, Yan-Ming Zhang, Cheng-Lin Liu, and Yoshua Bengio. Drawing and rec- ognizing chinese characters with recurrent neural network. IEEE transactions on pattern analysis and machine intelligence, 40(4):849-88) 01288,66,,,,,,, 2017.
Bocheng Zhao, Jianhua Tao, Minghao Yang. Zhengkun Tian, Cunhang Fan, and Ye Bai. Deep imitator: Handwriting calligraphy imitation via deep attention networks. Pattern Recognition, 104: 107:007080, 2020.
14

---

Published as a conference paper at ICLR 2026
APPENDIX SECTION
A Datasets and Preprocessing
15
Implementation Details of DiflInk
16
B.1 Details of InkVAE
16
B.2 Details of InkDIT
17
B.3 Training Configuration
18
B.4 Evaluation Metrics
18
More Experimental Results
19
C.1 Style Control and Consistency
19
C.2 More Comparisons with existing Methods
19
C.3 More Results Generated by Difflnk
22
C.4 DDIM Sampling Steps Evaluation
23
Discussion
23
D.I Comparison with Two-Stage Methods
23
D.2 Implementation of DiffInk
23
D.3 Applying to Other Languages
24
D.4 Improving OCR Performance with Synthetic Data
24
D.5 Personalized Handwriting Generation with DiffInk
25
D.6 Limitations
26
E Use of Large Language Models (LLMs)
26
A DATASETS AND PREPROCESSING
Dataset For Chinese TOHG, we use the CASIA-OLHWDB 2.0-2.2 (Liu et al., 2011) datasets, which contain over 50,000 online handwritten text lines collected from 1,019 writers. Due to training resource limitations, we select 500 writers based on the average length of their text lines, prioritizing those with longer and more informative content. Among them, 400 writers used for training and 100 for testing. As shown in Figure 9, we further analyze the text-line length distribution in the training set, where the average length exceeds 30 characters (higher than the reported 26.68 characters in prior benchmarks (Yin et al., 2013)) and the maximum length goes beyond 40 characters- sufffciiet to cover typical text- line writing scenarios in daily life.
1500
2500
1500
500
Length Text
Figure 9: Analysis of text-line length distribution in the Chinese handwriting dataset. Due to dataset constraints,
method supports up 52 charac- ters per line, which is higher than the average of 26.68 characters reported in (Yin et al., 2013) and sufficient for practical usage.
We observe that the character distribution in text-line data exhibits pronounced long-tail pattern. To mitigate this issue, we design frequency-awen data augmentation strategy.
15

---

Published as a conference paper at ICLR 2026
Specifically, we sample isolated character instances written by the same author from CASIA- OLHWDB 1.0-1.2, with sampling probabilities inversely proportional to their frequencies in the original text-line ayppus-assignnnn higher selection chances to rare characters. For each sampled author, we randomly select a text-line from the training set as a layout template, and place the sam- pled characters according to the original character BBoxes. This strategy enables the construction of synthetic text-lines that effectively compensate for the scarcity of low-frequency characters. In total, we generate approximately 67,000 text-line samples for training.
Data processing We first normalize the (z, y) coordinates of all trajectory points following prior work (Zhang et al., 2017), to address scale inconsistencies caused by variations in device resolution and handwriting styles. This normalization facilitates more stable learning of structural patterns in handwritten text. Subsequently, we apply the Ramer-Douglas Deae-Douugla-PPucccRRRDDDDD Peucker(RDP) (Douglas & Peucker, 1973) algorithm to each stroke sequence to remove redundant points, thereby reducing the overall trajectory length and improving computational efficiency. The simplification coefficients are empir- ically selected based on extensive trial experiments: we set them to 0.4 for CASIA-OLHWDB. The average sequence length of Chinese handwriting is reduced by about 500 points after simplification. These values balance preserving handwriting structure with noise reduction, while also shortening the input sequences. For each batch of text-line data, we pad all sequences to the maximum line length. The padding regions are filled with = y 0, while the pen-state pen is set to the End of Character symbol.
B IMPLEMENTATION DETAILS OF DIFFINK
B. DETAILS OF INKVAE
Reconstruction and KL Divergence As the latent modeling module of the latent diffusion Trans- former, Ink is primarily designed to produce compact representations that minimize reconstruc- tion loss. Specifically, InkVAE adopts ID convolutional encoder-decoder architecture with resid- ual connections. The encoder processes the input handwriting sequence of shape N x 5 through a series of downsampling layers, progressively transforming it into representations of size N/2 x 128, N/4 x 256, and finally N/8 x 384, forming a latent sequence with an 8x temporal compression ratio. The decoder mirrors this structure with residual ID convolutional blocks for upsampling, gradually reconstructing the sequence into N/4 4 x 384, N/2 x 256, and N X 123. The final out- put of the decoder parameterizes p = 20-component Gaussian Mixture Model (GMM) for (a. y) trajectory prediction, with the remaining dimensions used for pen-state classification (3 classes). A Transformer- based trajectory decoder (3 layers, hidden size 256, heads) employed to enhance sequence modeling. The overall reconstruction loss Liee consists of two components: the negative log-likelihood loss Lamm for coordinate modeling, and the cross-entropy loss Epen for pen-state clas- sification. A KL divergence loss is also applied to the latent space, as is standard in VAE frameworks, to promote regularization; to balance reconstruction quality, we follow diffusion-baas models and set its coefficient to a small value.
The Cemm is defined as Equation 4, where p. (xx, Me) the ground-truth point at timestep and each mixture component is parameterized by mean fmm.t. covariance Σm.t (often diagonal), and mixing coefficient Tm.t. The pen- pen-state prediction is treated 3-way classification task (pen-down) pen-up, end-of-stroke) with loss defined as Equation where v is the one-hot ground-truth label and PLk is the predicted probability for class k at timestep This focal loss formulation down- weights easy samples and focuses training on harder ones.
T
M
Eymm
log
Σ m==1
Tm, .N (pr Mm, Emei)
(4)
T
1=1
1
Chocal
Ork (1 p. vitt log Pt
(5)
T
Task relevant Regularization Recent studies (Tang et al., 2025; Yao et al., 2025; Guo et al., 2025) suggest that well-structured latent space plays erucial role in the effectiveness of downstream
16

---

Published as a conference paper at ICLR 2026
Algorithm InkDiT Training and Inference
Training Phase:
Inference Phase:
1: Input: Zo, Liet, Z
Input: 27 (O 1), Tnet,
2: Sample ~ (1,11) 1
2: for t ......I. do
3: Sample noise ~ (0, I)
3: io = InkDiT ([It, Tadl 7 1)
4: Compute 24 Vā + v1 - 6.
4: <= (xt vā= T0) v1
5: Predict šo InkDiTT (Te, Feet 71, f)
5: Jt-1 va--1 30
6: Update InkDiTa using loss: Lar = 1100 - moll?
end for
Final output: do
diffusion-based modeling. To this end, beyond trajectory reconstruction, InkVAE introduces two auxiliary objectives to impose task-specific regularization on the latent space: (1) A style classifier composed of an LSTM (Staudemeyer & Morris, 2019) with attention pooling and a 400-class writer identification head, and (2) a Transformer-bat bbased OCR module (hidden size 384) equipped with a 2648-clan class character classifier and trained with CTC loss to encourage content-aware representation learning. The style classification loss and the OCR loss are defined as Equation mer uses cross-entropy loss between the writer identity label yyyle and the predicted probability payle and the latter adopts the standard CTC loss (Graves al., 2006) to align predicted character sequences with ground-truth transcriptions.
where the for-
Luy
Vc
style
log style
Lour = CTC(P you).
(6)
c=1
B.2 DETAILS OF INKDIT
To generate text lines conditioned character sequences, InkDiT takes as input three components: the noisy latent representation Ie, style condition Zeef and the corresponding text embeddings. The text embeddings is processed by lightweight encoder to produce content-awarg features. These three inputs are then fused and projected into a unified joint space, which is subsequently modeled by multi-layer Transformer for time-aware latent denoising.
Content Encoder To model the character-level semantics in handwriting generation, we employ
a lightweight content encoder based on three stacked ConyNeXI-V2 (Woo et al., 2023) blocks. The input to this encoder is a sequence of padded character embeddings with a dimensionality of 512. Each ConvNeXt-V capture local context, followed by LayerNorm layer and a two-layer feedforward subnetwork with GELU activation and Global Response Normalization (GRN). Residual connections are applied to stabilize training and preserve lower-level features. This architectural design enables the encoder to effectively extract local and contextual dependencies among character tokens, resulting in content- aware features that are well aligned with the latent space representation used by the diffusion model.
block consists of depthwise ID convolution with kernel size 7 to
DIT Backbone The InkDiT backbone operates entirely within a 384-dimensional latent space. During each denoising step, it receives three conditional inputs: the noised latent representation It e 3 a style condition feature Jeef E ix344 and the content feature derived from the ConvNeXt-based encoder. These inputs are fused through mixing operation and subsequently projected to a 896-dimensional joint space using a linear layer. The resulting fused representation is then passed into a 16-layer Transformer model that follows the standard design of multi-head self- attention and feedforward networks. Each Transformer block incorporates adaptive normalization and gated residual connections modulated by timestep embeddings, enabling the model to perform time-aware denoising. This design facilitates joint reasoning over content semantics, spatial layout, and temporal dynamics within a unified generative framework.
The diffusion process follows cosine noise schedule with T = 1000 timesteps. Instead of predicting the noise, InkDiT is trained to directly estimate the denoised la- tent variable vo. following the DDPM-style parameterization. During inference, DDIM (Song et al., 2020) sampling with steps is used efficiently generate the final latent output.
Diffusion Process
The complete training and inference procedures are summarized in Algorithm 1. During training, the model learns to recover 2o from its noisy version I,, conditioned on both textual input and reference
17

---

Published as a conference paper at ICLR 2026
trajectory. To further improve generation quality, we apply lightweight DDIM-specific fine-tuning, where the model is supervised to predict žy at intermediate steps by unrolling a few DDIM iterations. This enhances fidelity with minimal computational overhead. In inference, generation starts from Gaussian noise at = T and iteratively denoises via DDIM updates, producing a coherent and stylistically aligned latent sequence that is ultimately decoded into an online handwriting trajectory.
B.3 TRAINING CONFIGURATION
For InkVAE, all components-includingg the sequence encoder, sequence decoder, and two regular- ization classifiers are jointly optimized with a learning rate of x 10-1 , ,3 = (0.9, 0.99), and a weight decay of 10-4, To stabilize training, we apply gradient clipping with a maximum norm of 5.0. For InkDiT, we adopt the same settings but use smaller learning rate of 7.5 x 10 and stricter gradient clipping norm of 1.0. both cases, the learning rate schedule combines 5% linear warm-up phase followed by cosine decay, implemented using the LambdaLR scheduler in PyTorch. All models are trained from scratch on 4xNVIDIA RTX A6000 GPUs. Training Ink VAE takes about 16 hours, whereas InkDiT training requires roughly 3 days. We employ the AdamW optimizer (Loshchilov & Hutter, 2017) for both stages of training.
B.4 EVALUATION METRICS
To evaluate the readability and correctness of generated text lines, we adopt Accurate Rate (AR) and Correct Rate (CR) as defined in Equations 7 and 8. Following (Liu et al., 2011), N₂ is the total number ground-truth characters; De. Se, and Le denote the numbers of deletion, substitution, and insertion errors, respectively. Specifically, we train text-line OCR recognition model solely on the training set, which still achieves high accuracy of 97.62% when directly evaluated on the test set. The model adopts the encoder of inkVAE as the backbone, followed a character classification head, and is subsequently used to evaluate the textual accuracy of generated handwriting samples.
AR (Ne D. -S. 1o) 100% N₂
(Ne De Se) N
CR
100%
(8)
The text-line DTW (Berndt Clifford, 1994) distance is computed over the (x, y) coordinate se- quences of each full text line and normalized by the length of the corresponding ground-truth tra- jectory, following the formulation in Equation 9. Here, N is the number of test samples, |o:"" denotes the length of the i-th ground-truth trajectory, and DTW(.) is efficiently computed using the t torch- fast dt w library.
Norm-DTW DTW (z.ft pred
The style score is obtained by feeding the generated text lines into style classifier, predicting their corresponding writer identities. this end, we train a 100-class writer identification model on the testing set, which achieves classification accuracy of 99.9% on the test set, demonstrating its strong discriminative ability for style evaluation. The style classifier consists of the encoder of InkVAE followed by writer classification head. Besides, generation efficiency (Avg. char/s) is evaluated as the average number of characters generated per second. All methods are tested with a batch size of on the same NVIDIA RTX A6000 GPU. The average generation speed is computed by dividing the total number of generated characters by the elapsed time, excluding data loading and other preprocessing operations.
18

---

Published as a conference paper at ICLR 2026
Writers1
1R,
A) $ 80, 2 d. Ke
sel
Wyiter-2
Rb. EA. -2
66.2444 1A 2.
R 2
8, 2 45
Writer-3
- -u
* R.68888888000000 36
Writers
1545.
62.4 2 &
Text Line
"MAA 2- 48 Mill NE
"A 3A BAAKE + FO
BUU de B
Figure 10: Same text across different styles. Generating the same text in different styles shows that DiffInk captures distinct style variations while maintaining intra-writer consistency.
Stylc-1
Reference
Diffink
Text
2b
41 NE
Style-2 o
Stylo-3
Reference
Diffink
Style4
B
Text
Di AA 0U
Style-S
B
Reference
Diffink
Text
Style-6
1
Figure 11: Multi-style input. DiffInk can take two style references simultaneously and, conditioned on the given text, generate handwriting that blends characteristics of both styles, enabling control- lable style manipulation.
C
MORE EXPERIMENTAL RESULTS
C.I STYLE CONTROL AND CONSISTENCY
Style Consistency As illustrated in the Figure 10, we showcase generated text lines from four writers the test set. Each writer conditioned on the same textual content to generate handwrit- ing samples in their respective styles. Horizontally, despite the variation in content, the generated samples from the same writer exhibit strong stylistic consistency. For example, writer consistently produces slanted layouts from the lower left to the upper right. Vertically, given the same content input, samples generated in different styles show clear variations in both layout and writing traits, such overall length, character size, and spatial arrangement.
Style Control DiffInk capable of handling style-mixing inputs, allowing the model to generate handwriting that interpolates between multiple reference styles. As illustrated in the Figure 11, given two style examples and a target text string, the model synthesizes pen- cen-trajectory sequences that simultaneously reflect structural consistency with the textual content and stylistic characteristics drawn from both references.
C.2 MORE COMPARISONS WITH EXISTING METHODS
We provide additional comparisons with existing approaches, including Drawing (Zhang et al., 2017), Deepiminator (Zhao et al., 2020), WLU (Tang & Lian, 2021), SDT (Dai al., 2023), and
19

---

Published as a conference paper at ICLR 2026
OLHWG (Ren et al., 2025). As illustrated in Figure 12 and Figure 13, method delivers superior text-line generation quality over a broader spectrum of handwriting styles.
Writer Style
Text
FF r-
Drawing
Hokee fee 1 55----56
Deep
*6 +79
Iminator
WLU
* 4,49 Hww
A #
SDT
* 4:A0 Hala
OLHG
* Hu
DifMkk (Ours)
Hait M /
#
Target
*
* Spp
Asp VEGER
Writer Style
Text
Car lang
Drawing
Deep
hyp 465 _,80
Car lavg f 1
Iminator
WLU
Tpp $M
MM
Cav levss a
SDT
5,0 46 .al
Cor lorg 9.
OLHWG
8. 46 RR a
Cer kwng t
DiffInk
5, & S
Cev long & 14
(Ours)
Carkayy 7
Target
Writer Style 3
Text
(20:5 #FAD Faea
Drawing
NgC 1
Deep Iminator
WLU
SDT
OLHWG
Diffink
(Ours)
Target
Figure 12: More visual comparisons with SOTA methods. DiffInk generates more natural text- line results, while the red boxes highlight unnatural character connections.
20

---

Published as a conference paper at ICLR 2026
Writer Style
Text
*4 2f 34
Drawing
Deep Iminator
WLU
SDT
*l
OLHWG
DimMk (Ours)
Target
Writer Style
Text
Drawing
18 1M6
Deep
2 1/%6 4
Iminator
WLU
# 1M/
$ #
OLHWG
DiffInk (Ours)
x #
Target
$ 13
Writer Style 6
Text
Drawing
Doep
Iminator
WLU
SDT #
OLHWG
Diffik (Ours)
Target
Figure 13: More visual comparisons with SOTA methods. DiffInk generates more natural text- line results, while the red boxes highlight unnatural character connections.
21

---

Published as a conference paper at ICLR 2026
C.3 MORE RESULTS GENERATED BY DIFFINK
In Figure 14, we present additional results generated by DiffInk. We randomly sample eight writers from the test set and generate multiple text lines for each of them. Visual inspection reveals that the generated text lines from the same writer exhibit consistent local stroke patterns and global layout styles, while clear stylistic differences emerge across different writers. These results demonstrate that DiffInk effectively captures inter-writed style variation, and that our lightweight style classifica- tion strategy improves the model's ability generalize across styles. Moreover, the generated text lines also maintain high content accuracy and readability, further validating the effectiveness of the proposed DiffInk framework.
Writer Style
Writer Style 2
Text: BAAARE
Text: F Mr MEAK 2n
Reee
A #e #
Tex:: ::
Text: 3
H.E. 44
Text: 1
Tex:: 47
M& 7:4 4
Text: E/P0
Text:
" s
Writer Style
Writer Style 4
Text: 51
Text: H. 10
H!!
5 5
Tex:: 624
Tex::
Text:
Text: .
Text: BR
Text: M. 4.
de B
Writer Style 5
Writer Style 6
Text: Mile e.
Text: 2A
Text: #
Text:
Text:
Tex::
Text:
Tea::
Writer Style?
Writer Style 8
TENt: 1
Text:
Tea::
Figure 14: More generation results by DiffInk (Ours). DiffInn is capable of producing diverse handwritten long text lines, exhibiting high consistency within the same style and clear distinctions across different styles.
22

---

Published as a conference paper at ICLR 2026
Table 4: DDIM sampling-step analysis of Difflnn, illustrating the trade-off between generation quality and inference speed.
Sample Steps
AR% t
CR %
Style
DTW
Time Cost
36.23
36.39
35.85
1.028
0.2x
2
79.70
80.00
66.31
1.022
0.4x
89,93
90.19
73.51
1.026
0.6x
4
93.16
93.39
76.47
1.040
0.8
5 (Ours)
94.38
94.58
77.38
1.052
1x
6
94.86
95.01
77.69
1.058
1.2x
10
95.35
95.52
78.32
1.067
2.0x
15
95.36
95.56
79.10
1.068
3.0x
20
95.27
95.47
79.16
1.076
4.0x
C.4 DDIM SAMPLING STEPS EVALUATION
To evaluate the impact of DDIM sampling steps on generation performance, we conducted an ad- ditional experiment. shown the table 4, We observe that the performance of DDIM sampling continues to improve as the number of steps increases from 1 to 5. However, when the number of steps exceeds 10, the gains content and style scores become marginal, while the DTW value increases and the sampling cost grows significantly. Therefore, we adopt 5-step DDIM sampling in this paper as a practical trade-off between generation quality and computational efficiency.
D DISCUSSION
D.1 COMPARISON WITH TWO-STAGE METHODS
As illustrated in Figure 15, conventional two-stage methods decouple character generation from layout prediction and rely on additional post-processing to stitch characters together, which often results in less coherent text lines. In contrast, our proposed DiffInk models content, style, and layout within unified framework, enabling genuine end-to-end generation that produces text lines with more natural character connections and globally consistent layouts.
Stage 1: Handwriting Font Generation
Stage 2: Text Line Layout Modeling
Charucter-by by-Chruucterrr Font Generation
(a)
Style reference
Character Handwriting
Prodicted LSTM
post-processing
Characters:
22
Text:
Mi 51
Ours One-Stage Text Line Handwriting Generation
(b)
Style Reference
' g a seaeoo fatk 3
Diflnn(OOur))
Figure 15: Comparison with character-layout decoupled approaches: (a) two-stage pipeline combining handwritten font generation with layout post-processing; (b) DiffInk, which takes text and style reference directly output complete text lines. Unlike the two-stage pipeline, Difflnk generates more natural character connections rather than mechanically stitching bounding boxes.
D.2 IMPLEMENTATION OF DIFFINK
In the main experiments, reference, both for convenient sampling and to evaluate whether the model can generate layouts that follow the trend of this given reference. This setting is adopted for convenient comparison with
continuous reference segment from each text line the style
23

---

Published as a conference paper at ICLR 2026
A anoft retm, Ymmnn t snitter poom Sines
it Shimodor Rald Wim. Ishmodeus dold him "Ashmodeus told him"
Seurth gour livn thim jonndh gour hur his "Smith gave nim his" Blad mayy lun ovr Aach of Dondth snddng Vioo owne Raate of Baro. blow your Shiddes dff "sure your whiskers off
That weiher sichy, bands that neither shicyy loads
"w another room Simon
neither sticky bends"
Idee dns Boegingg see
Boynnn dod to his Santh,
Sura Mer your whiccers 1
Sobout herr The play
Phorons Alsher spact.
RRichar Seaver. The play way
"theme dear to his heart."
"and only his DwN lands of
Figure 16: Results the IAM test set. Difflnk generates English text lines with consistent style and accurate content. The top row shows generated results, and the bottom row shows the corre- sponding target samples.
Recognition performance on ICDAR13 (Yin et al., 2013) with and without synthesized data. Adding synthetic text lines improves AR and CR by 8.5 and 8.6 points, demonstrating both the effectiveness and quality of the generated data.
Table 5:
OCR Model
Training Data
CR f
f
Real data only (Baseline)
83.9
84.2
CNN+Transformer
+ DiffInnk Synth (Ours)
89.6
90.0
Real data only (Baseline)
86.2
86.6
GLRNet (Peng et al., 2021)
+ Diffllnk Synth (Ours)
90.9
91.1
recent SOTA methods (Ren et al., 2025), which feed the layouts of the first ten characters of a real text line into the model; the model then predicts the positions and sizes of subsequent characters based the reference layout, the input character content, and average character-size prior. To ensure a fair comparison, we instead provide the continuous handwritten trajectory segment of the real text line-containing both style and layout information-as input to our model. For the other compared methods, the same reference segment is used as the style condition. The corresponding character bounding boxes of this reference fed into the OLHWG layout prediction module as prompts, and the final text-line results are obtained by scaling the generated characters according the predicted bounding boxes. Finally, line-level metries including content scores, style scores, and DTW distance-an are computed with each text line conditioned on its own reference segment.
D.3 APPLYING TO OTHER LANGUAGES
To validate the generalizability of the proposed method, we directly applied it to English text line generation tasks and conducted experiments on the IAM-OnDB dataset. The data process- ing pipeline remained consistent with the Chinese setup, including the use of the RDP algorithm (with the redundancy removal parameter set to 0.5) for redundant point removal and normalization of trajectory coordinates. The IAM-OnDB (Liwicki & Bunke, 2005) dataset contains approximately 10,000 handwritten English text line samples produced by 221 writers, featuring substantial cursive writing. We randomly selected data from 25 writers the test set, with the remaining data used for training. During training, all hyperparameters remained consistent with the Chinese experimental configuration, except for the learning rate, which was reduced to 5 x 10-5
Figure 16 showcases several examples of DiffInk applied to English handwriting generation. The model effectively generates text-line sequences that are both content-accurate and style-consistent, even when the handwriting features strong cursive connections and diverse stylistic variations. These results highlight the versatility and generalizability of the DiffInk framework across multilingual and stylistically complex handwriting scenarios.
D.4 IMPROVING OCR PERFORMANCE WITH SYNTHETIC DATA
In practical applications, the quantity of handwriting data plays a critical role in improving OCR per- formance. To assess the impact of our synthesized data, we use the generated text lines to augment the original training set. For recognition, we adopt the same OCR model employed in this work for evaluation CNN+Transfomm tion, we also follow the recognition model design used in GLRNet (Peng et al., 2021) and conduct experiments under the same settings.
architecture the text-line recognition backbone. In addi-
24

---

Published as a conference paper at ICLR 2026
Our experimental results the ICDARI3 dataset (Yin et al., 2013) are presented in Table 5. It can be observed that the proposed generation method significantly improves the performance of the recognition model. Specifically, after augmenting the training set with our synthesized data, the AR and CR metrics increased by 8.5 and 8.6 percentage points, respectively. When using the recognition model from GLRNet (Peng al., 2021), our synthesized data further yields average improvement of around percentage points. This demonstrates that our synthesized data can effectively mitigate recognition degradation caused by the long-tail distribution of characters and the limited diversity of handwriting styles, thereby substantially enhancing the representational capacity of existing datasets and further confirming the high content and style quality of our synthetic data.
D.5 PERSONALIZED HANDWRITING GENERATION WITH DIFFINK
As text-line handwriting generation model, DiffInk (Ours) is useful not only for producing high- quality synthetic data for OCR systems, but also for broader generative applications such as per- sonalized handwriting services and human-computer interaction. For example, in a personalized handwriting generation scenario: for given writer, only single reference line- or even short trajectory of fewer than ten characters is provided as the prompt X with its corresponding text Tiet The user then specifies target text line gen and the model generates the correspond- ing trajectory X ggen- As shown in Figure 17, DiffInk yields text lines with coherent layouts under this one-shot setting. To further enhance diversity, the reference trajectory X,et can be augmented to simulate different structural layouts. Furthermore, our approach offers a promising direction for paragraph-level generation. As illustrated in Figure 18, we present an example of handwritten paragraph of classical Chinese poetry. By simply concatenating the line-level results, complete paragraph can be obtained, further demonstrating that DiffInk can support broad range of handwriting-relhee applications.
#
Reference
A E.
Text
Generate
Diffink
* *
t:4
3i 1 HERE
rit M4
Reference
3 3
Text
Generate
Difflnk
4 4 B9...
7, 76
2 1
Reference
7
1
200044
Text
Generate
8 2 7 111b
4/
Diffink
48
+ 41
Figure 17: One-shot generation results. DiffInk is capable of producing text lines of arbitrary length conditioned only on a short reference trajectory, supporting arbitrary character combinations from the defined vocabulary. Even under this one-shot setting, the generated lines maintain both high character accuracy and consistent writing style. Different colors denote different styles.
25

---

Published as a conference paper at ICLR 2026
RĀG TX B
A N
Reference
#
01 K
AA 4 N
Text
Generate
Diffink
MEN M2. A
T #
paragraph classle Chinese poetry: Xing (The Song the Pipa), Tang Dynasty, Bai
Extending handwriting generation the paragraph level, we take classical Chinese poetry as an example. Given style reference, the text is input line by line to produce line-level results, which are then concatenated to form a complete handwritten paragraph.
Figure 18:
D.6 LIMITATIONS
While our model establishes SOTA performance on the notoriously difficult task Chinese hand- written text-line generation, direct application to other languages (e.g.. Latin-based scripts) still requires retraining. Exploring how to develop a unified multilingual generation model with only minor modifications will therefore be one of our key directions for future work.
E USE OF LARGE LANGUAGE MODELS (LLMS)
LLMs, such as ChatGPT (OpenAI, 2023) and DeepSeek (DeepSeck-Ae AI, 2024), were employed solely for language polishing and minor writing refinements in this paper. They were not used for generating ideas, designing methods, conducting experiments, analyzing results, or creating figures.
All scientific contributions, including the conception of the research problem, model development, and experimental validation, were carried out independently by the authors.
26


Reviews:
[{"review_id": "bsWQftAWP3", "reviewer": "Reviewer_p6M8", "summary": "This paper addresses the challenge of text-to-online handwriting generation (TOHG), focusing on full-line handwritten Chinese text. It introduces a latent diffusion Transformer framework for full-line handwriting synthesis, featuring InkVAE for latent-space modeling and InkDiT for precise pen trajectory generation. Experiments show that DiffInk surpasses current SoTA methods in both glyph accuracy and style fidelity for TOHG.", "questions": "1. InkDiT mainly performs diffusion on latent representations, and thus DiffInK heavily relies on the performance of VAEDecoder in InkVAE. The question is whether InkVAE can generalize to unseen writers and text corpora, especially for full-line text generation?\n\n2. Although InkVAE can learn glyph- & style-aware features, the style and glyph information of each handwriting sample are mixed or entangled in latent presentations (extracted by InkVAE). In other words, the style features and textual contents are entangled, and only paired styles & texts can successfully reconstruct the input handwriting sample. Therefore, the concern is whether the X_ref and Content Z in Fig. 3 must attain the overlapped textual contents? Whether the model capable of generating accurate handwriting samples if the textual contents of X_ref and Content Z are completely disjointed?\n\n3. Different from the auto-regressive model that explicitly produces the end-of-sequence, how does the DIffInk know the lengths of generated full-line handwriting trajectories?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. This paper proposes the glyph- & style-aware regularization to preserve character-level structural information and enhance the discriminability of writing styles. The introduced InkVAE can learn more discriminative and representative latent features.\n2. The proposed InkDiT performs the diffusion process on the latent space, which is more efficient and effective than previous Diffusion models for TOHG.\n3. Experiments show that the proposed DiffInk outperforms the previous SoTA models for TOHG with higher glyph accuracy, style fidelity, and generation efficiency.", "weaknesses": "1. The competing models in Table 1 are slightly few, with only three different models. More competing models for OHG should be included, such as SketchRNN.\n2. Although TOHG is less investigated, the generation paradigm of DiffInk is very similar to the existing latent generation model StableDiffusion.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pveA6coq2E", "reviewer": "Reviewer_Kr6c", "summary": "This paper proposes a latent diffusion Transformer for online handwriting generation. Authors cite speed and the ability of conditioning on the  full line as the main problems addressed.\n\nTwo main components are:\n* InkVAE: VAE trained with additional losses for recognition (forcing generated samples to be recognizable) and writer-identity-style classification (enforcing style-space disentanglement).\n* InkDiT: diffusion Transformer (DiT) in the latent space created by InkVAE, conditioned on the embedding of a target text and style reference passed through InkVAE encoder.\n\nExperiments are done on CASIA-OLHWDB (Chinese) dataset, with evaluation in terms of accuracy, style consistency, and generation speed.", "questions": "- Model implicitly learns layout from the style reference. Have you considered a more explicit form of control, such as conditioning the InkDiT on a target baseline curve, bounding boxes, or other spatial parameters?\n\n- The paper claims 800x speedup over the baseline. Does 58.47 char/s figure for DiffInk include the conditioning, the 5-step DDIM sampling and the final VAE decoding step?\n\n- Can the model currently handle OOV characters in any way?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "* InkVAE design is interesting, novel, and well ablated (ex. Table 2, Figure 2 ablate the auxiliary losses for InkVAE construction). The approach seems generalizable (ex could be similarly applied to cursive handwriting, sketch generation, etc.) \n\n* Quantitative results are quite strong (most impressively in the generation speed, but also on recognition accuracy and style classification). While two-step approach (first generate bounding boxes, then fill the letters) could also allow for high generation speed (as individual characters could be generated in parallel, at least in the case of Chinese), the proposed one-step approach with InkDiT would be superior in cases like cursive and sketches.\n \n* Additional results & ablations (generalizability to English handwriting,use as data augmentation tool for OCR, detailed comparison to two-stage approach) further strengthen the work (in my opinion, they should be more prominently highlighted in the main body - before reading them in the appendix, I assumed the model would generalize poorly beyond Chinese handwriting.\n\n* Assuming the code will be released beside the paper, I believe this will be a significant contribution on which the community can further build.\n\n* The writing is generally very clear, and the set of ablations is comprehensive.", "weaknesses": "Limited evaluation: Main evaluation is performed on a single Chinese dataset. Paper could be significantly strengthened by performing evaluation on other domain, such as other languages (BRUSH, IAM), documents (IAMOnDB), sketches (QuickDraw), math (MathWriting), etc.\n\nStyle Conditioning: The model is conditioned on a single style vector $x_{ref}$. It is unclear how it would handle more complex or ambiguous style inputs, such as multiple, varied reference samples from the same writer, or a single reference that contains two different styles. Paper could be strengthened by the evaluation of style mixing, etc.\n\nText conditioning: The content encoder relies on a learnable codebook for a fixed set of characters. This is a standard approach but presents a hard limitation for out-of-vocabulary (OOV) characters, which is a significant issue for languages like Chinese (esp given ~2.6k characters while the alphabet could expand to 30k). Paper could be strengthened by the discussion around this topic, or by evaluation of image-based conditioning for text.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xPZalVjr3k", "reviewer": "Reviewer_U8jE", "summary": "The paper presents a well-executed study with significant contributions to the task of end-to-end handwritten text-line generation, demonstrating both theoretical and practical advancements. While the empirical results are strong and the methodology is clearly justified, there are certain limitations in the experimental setup, data synthesis approach, and the exploration of real-world implications. Addressing these concerns would strengthen the work and provide a more comprehensive understanding of the method's impact and limitations.\n\nIf the authors can address my doubts or point out areas I may have overlooked, I would be willing to reconsider my evaluation and potentially improve my score.", "questions": "Please refer to the weakness.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. **Significant Empirical Advancement in a Challenging Task**  \n   The paper demonstrates a notable and convincing empirical improvement on the relatively underexplored and complex task of end-to-end online handwritten text-line generation. The proposed DiffInk framework achieves superior performance across both quantitative metrics (Content AR/CR, Style Accuracy, DTW) and qualitative visual comparisons, thereby setting a new state-of-the-art benchmark in this domain.\n\n2. **Clear, Well-Structured, and Justified Methodology**  \n   The overall framework is presented with exceptional clarity. The design of the two core components, InkVAE and InkDiT, is both logically sound and well-motivated. The introduction of task-specific regularization losses (OCR, style) to structure the latent space offers a direct and effective solution to a clearly identified problem, making the method both easy to follow and reproducible.\n\n3. **Comprehensive and Convincing Ablation Studies**  \n   The paper is notably supported by thorough ablation studies that systematically validate the contribution of each key component (e.g., InkVAE regularization losses, InkDiT's content encoder). These experiments provide strong evidence that the observed performance gains are directly attributable to the proposed architectural choices.\n\n4. **Noteworthy Practical Efficiency**  \n   In addition to its quality, the method achieves a dramatic speedup in generation (e.g., >800x faster than OLHWG), making it not only more effective but also significantly more practical for real-world applications.", "weaknesses": "1. **Incremental Nature of the Core Technical Contribution**  \n   While the empirical results are robust, the core technical innovation can be characterized more as a skillful integration and refinement of existing components (VAE, DiT) rather than a groundbreaking algorithmic advancement.\n\n2. **Fundamental Limitation in Training Data Synthesis**  \n   The reliance on a data augmentation strategy that stitches isolated characters into text lines presents a significant methodological limitation. This approach inherently fails to capture the cursive connections and natural co-articulation found in real handwritten text, which likely imposes a ceiling on the model’s ability to generate fully authentic, fluid text lines and raises questions about the underlying sources of its performance gains.\n\n3. **Potential Bias in the Experimental Setup**  \n   The decision to compare the method against character-level models extended by a single, shared layout prediction module (from OLHWG) is a practical but potentially unbalanced choice for the baselines. This setup may unfairly attribute the weaknesses of this specific layout module to the baseline generation methods, thus exaggerating DiffInk's advantages in terms of coherence and layout.\n\n4. **Limited Exploration of Practical Impact and Downstream Utility**  \n   The paper would benefit from a more comprehensive discussion of its real-world implications. While the OCR data augmentation experiment is a step in the right direction, it uses a relatively weak baseline model, which diminishes the argument that the synthetic data is of sufficiently high quality to improve modern, high-performance systems.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "mR7tqyNWlX", "reviewer": "Reviewer_7DmP", "summary": "The paper introduces DiffInk, a framework for text-to-online handwriting generation (TOHG) that generates full-line pen trajectories conditioned on input text and a reference handwriting style. DiffInk is based on latent diffusion, consists of two core components: InkVAE and InkDIT.", "questions": "1. Regarding the evaluation metrics, the handwritten-line recognition model and the writer-identification model both employ the encoder of the proposed InkVAE as their backbone. Is this the exact network obtained during the training of InkVAE, or was an additional network trained separately? \n\n2. What loss function is used for the writer-identification model? Moreover, given that the training and test sets correspond to disjoint writer classes, why is it valid to perform inference on the test set after training solely on the training set?\n\n3. The method employs data augmentation to enlarge the training set by selecting individual characters from a single character dataset and inserting them into the existing line layouts. Since the synthesized characters appear disconnected, this disrupts the connections, seemingly contradicting the argument made in Section 4.5. What would the performance be if this form of data augmentation were removed? \n\n4. The comparison of generation speeds needs further clarification. For each method, what batch size was used? Moreover, for the prior diffusion based approaches, does the observed acceleration stem mainly from DDIM’s faster sampling schedule or from the lower dimensionality of the latent space? In addition, what is the relationship between the number of DDIM inference steps and the resulting generation quality?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1.  The application task is rare yet interesting, yielding strong results in both visual and quantitative evaluations. \n\n2. The method is clearly presented and easy to follow.\n\n3. Each sub-module’s motivation is well justified, and ablation studies are provided.", "weaknesses": "1.  Although commendable performance has been achieved on the specific task, the overall framework more closely resembles an assemblage of existing techniques, limiting its degree of novelty. Specifically, reference [1] already employs a latent diffusion model to generate handwritten text lines, the present work differs mainly in the data format (offline or online) and the way how textual content is encoded. As for the content-encoding strategy, reference [2] likewise adopts a character embedding combined with a lightweight content encoder.\n\n2. There are relatively few baselines, partly because prior work on full-line text generation is scarce. However, given the abundance of baselines for single-character generation, I believe this task can be viewed as a special case of line-level generation and constitutes a valid evaluation setting for assessing fine-grained handwriting style imitation.\n\n3. Several implementation details in the experiment section can be stated more clearly, please refer to the questions (as well as some issues).\n\n\n\n----------\n\n[1]. Template-guided cascaded diffusion for stylized handwritten Chinese text-line generation.\n\n[2]. Content and style aware generation of text-line images for handwriting recognition.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)