# N-gram Injection into Transformers for Dynamic Language Model Adaptation in Handwritten Text Recognition

Florent Meyer, Laurent Guichard, Yann Soullard, Denis Coquenet, Guillaume Gravier, Bertrand Coüasnon

► To cite this version:

Florent Meyer, Laurent Guichard, Yann Soullard, Denis Coquenet, Guillaume Gravier, et al. N-gram Injection into Transformers for Dynamic Language Model Adaptation in Handwritten Text Recognition. 2026. hal-05537580

HAL Id: hal-05537580

https://hal.science/hal-05537580v1

Preprint submitted on 5 Mar 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
arXiv:2603.03930v1 [cs.CV] 4 Mar 2026

n-gram injection into transformers
for dynamic language model adaptation
in handwritten text recognition

Florent Meyer¹,²✉ [0009-0001-7527-5215], Laurent
Guichard¹[0009-0008-7853-8704], Yann Soullard²[0009-0001-8048-2489], Denis
Coquenet²[0000-0001-5203-9423], Guillaume Gravier²[0000-0002-2266-5682], and
Bertrand Coüasnon²[0000-0002-7077-0751]

¹ ANTAI, Rennes, France

² Univ Rennes, CNRS, IRISA - UMR 6074, Rennes, France
florent.meyer@irisa.fr

**Abstract.** Transformer-based encoder-decoder networks have recently achieved impressive results in handwritten text recognition, partly thanks to their auto-regressive decoder which implicitly learns a language model. However, such networks suffer from a large performance drop when evaluated on a target corpus whose language distribution is shifted from the source text seen during training. To retain recognition accuracy despite this language shift, we propose an external *n*-gram injection (NGI) for dynamic adaptation of the network's language modeling at inference time. Our method allows switching to an *n*-gram language model estimated on a corpus close to the target distribution, therefore mitigating bias without any extra training on target image-text pairs. We opt for an early injection of the *n*-gram into the transformer decoder so that the network learns to fully leverage text-only data at the low additional cost of *n*-gram inference. Experiments on three handwritten datasets demonstrate that the proposed NGI significantly reduces the performance gap between source and target corpora.

**Keywords:** Language model · Transformer · Knowledge injection · Linguistic bias · *n*-gram · Handwritten text recognition

# 1 Introduction

In an era of digitalization, handwritten text still plays a crucial role in our daily lives as not all tasks and procedures have been dematerialized. For instance, a variety of handwritten content can be found in historical records, administrative forms, personal documents or complaints addressed to companies. Handwritten text recognition (HTR) is the computer vision discipline aiming at transcribing scans of these handwritten documents to machine-readable text. The recognition is still challenging as of today because of variations in writers' style, ambiguities arising from cursive scripts, or syntactic and grammar rules that
differ from one language to another. Nowadays, transformer-based [40] encoder-
decoders [2,7,22,26] achieve remarkably low transcription error rates in HTR
thanks to an auto-regressive modeling of the language at training time.

When designing HTR systems, it is most often assumed that the training and
test datasets follow distributions which are independent and identical [10]. How-
ever, real-world use cases do not always satisfy this constraint. Consequently,
overlooking the language *shift* between corpora can lead to a significant recog-
nition accuracy drop [3,18,28], making the recognizer unusable on an out-of-
distribution corpus. Our experiments on state-of-the-art encoder-decoders con-
firm the performance degradation of transformers on a test corpus whose lan-
guage distribution differs from training.

In this paper, we propose *n*-gram injection (NGI) as a domain adaptation
method to reduce the performance gap between a training dataset, referred to as
*source*, and a *target* dataset, i.e. that seen at test time, whose language distribu-
tion is shifted. NGI is a lightweight method for guiding a neural recognizer with
an external *n*-gram language model (LM). By injecting *n*-gram distributions into
a new lightweight auto-regressive FCN-transformer, the word attention network
(WAN), we let the network learn to adapt to the external LM. Most importantly,
our method allows the user to dynamically switch the *n*-gram at test time to
better match the shifted language distribution of a target corpus. Therefore, bias
and the associated performance loss are mitigated without any extra training on
target images.

To match the shift naturally present in an industrial form recognition task,
we provide our own splits of well-known HTR datasets where lexicons or char-
acters sequences differ between source and target corpora. Extensive evaluation
reports large gains on target subsets while preserving performance on source
subsets, demonstrating that NGI alleviates bias towards the source language.
While evaluated on word-level HTR, the proposed *n*-gram injection framework
could be applied to any architecture relying on an auto-regressive character-level
transformer decoder.

2 Related work

2.1 Bias in handwritten text recognition

Although remarkably low error rates are obtained on printed text nowadays,
HTR remains a challenging research problem as handwriting is arguably more di-
verse and complex than printed text. Early on, recurrent neural networks (RNN)
and more particularly long short-term memory (LSTM), trained thanks to the
connectionist temporal classification loss [11], have been widely adopted by the
community of document analysis. The bidirectional LSTM [12,31] and multi-
dimensional LSTM [4,13] are notable examples of such recurrent architectures.
Fully convolutional networks (FCN) then appeared as a means to overcome the
latency caused by the sequential nature of RNN [6,45]. Departing from recur-
rent and convolutional architectures, transformers [40] equipped with attention
mechanisms implicitly build language modeling thanks to their auto-regressive decoding process, therefore achieving remarkable performance in HTR [2,22,26]. Following this trend, the DAN [7], combining a CNN encoder with a transformer decoder, has become the reference for HTR on full-page scans.

Despite their high performance, those neural networks (NN) model source language seen during training and thus suffer from a performance gap when evaluated on a target corpus whose language is shifted from the source, as shown by diverse work in scene text recognition [16,27,42,46,47] and HTR [3,18,28]. Yet, Garrido-Munoz & Calvo-Zaragoza [10] relate that current research on network architectures is mostly oriented towards increasing performance on standard benchmarks, making the strong hypothesis of an independent and identical distribution between source and target corpora. However, this is overoptimistic in real-world use cases. They demonstrated that the difference between source and target character error rates reaches an astounding 37.6 points averaged on seven widely-used HTR datasets.

## 2.2 Combining language models with neural networks

**Language model variants** LM can be categorized into two families, namely statistical and NN LM. On the one hand, *n*-grams are probabilistic models assuming that the next token in a sequence depends on a fixed-sized window of the *n* − 1 previous tokens [17]. Estimated using token counts, they provide a probability distribution over all possible tokens, making use of smoothing algorithms to avoid giving a zero probability to unseen sequences. Used in various natural language processing (NLP) tasks for several decades, *n*-grams have been the reference for the post-processing of the optical recognition in HTR [31,36,41]. On the other hand, external NN LM have the key ability to capture long-range relationships between tokens. However, the training and inference of a neural LM comes at a much higher cost due to its number of parameters, while *n*-grams suffice to achieve satisfactory performance on diverse use cases. Hence, they still are a popular choice in recent work [23,36].

**Implicit language modeling** Numerous forms of knowledge injection were proposed to provide NN with information about the structure of the input tokens sequence. To better model the strong correlations between adjacent elements in music sequences, Tian *et al.* [38] group music-specific tokens before embedding them and sending them to a masked-language-modeling transformer encoder. In a similar manner, text tokens can be grouped based on 2-grams [29] or a word lexicon [8,9] to encourage the network to use these related tokens together. Conversely, some work explores the reinforcement of the implicit language modeling of an NN by predicting groups of tokens [5,37]. Similarly in pure NLP, an *n*-gram masking method enhances the integration of language information into masked language modeling by combining three prediction objectives, namely predicting *n*-gram identities as well as each separate token composing the *n*-gram, and detecting tokens replaced by a generator [44]. Such models require vast amounts of
data and training time since they are entirely NN-based, and HTR approaches
would require annotated text-image pairs to counteract language shift.

Explicit language modeling

*Pre-processing* A family of NLP methods proposes to accelerate the costly auto-regressive inference step of transformer decoders by refining a preliminary draft issued by a faster model. The draft, which can be seen as a pre-processing step, is a tokens sequence suggested either by a standard *n*-gram [20] or an LLM [34]. The main model, a larger transformer, then verifies and corrects the suggestions, delivering speed gains whenever low-cost draft tokens are deemed valid. As opposed to sampling discrete tokens from the preliminary model, we argue that keeping the empirical distribution over all tokens provides the main network with richer information.

*Post-processing* Numerous approaches post-process the NN output lattice *via* re-scoring with the Viterbi algorithm [31] or beam search [35,36] on characters, sub-lexical units or words. Notably, re-scoring with *n*-grams was shown to improve HTR performance of a transformer model [36]. Unfortunately, re-scoring computations explore large search graphs or lattices which induces a time complexity overhead. Also, post-processing does not let the NN learn to make wise use of the LM, which leads us to think there may exist better ways of combining the two. Indeed, post-processing is applied with fixed hyper-parameters regardless of the input image, while we suppose that an NN could adjust the integration of the LM depending on visual ambiguities or in the presence of a deteriorated scan, for instance.

*Layer injection* Some approaches propose the injection of external LM scores to be processed by an NN model. Examples include external LM based on RNN [15,18] which, as mentioned previously, require large amounts of training data and computing power. Furthermore, late fusion [15] of the LM does not allow the NN to fully exploit it, and the vision-language co-adaptation proposed by Kang *et al.* [18] cannot be learnt on a target domain without text-image pairs. As opposed to Li *et al.* [21], our goal is not to concentrate the network's language modeling effort on difficult cases and long-distance dependencies, but precisely to restrict implicit language modeling which would restrain language adaptation capabilities. Moreover, for each different domain, the same *n*-gram is seen during both training and testing, while our objectives is to allow switching to an *unseen* *n*-gram. The integration of *n*-grams within transformers remains mostly unexplored [36], arguably due to the supposedly satisfying language modeling of the auto-regressive architecture. In this work, we try to fill this gap by introducing the first language adaptation strategy for HTR relying on *n*-gram injection.
## 3 Context & problem statement

Recently, encoder-decoder architectures based on transformers [40] have achieved remarkable performance in HTR [2,7,22,26]. However, as shown later on and already demonstrated in the literature [3], they are prone to bias which causes a significant drop in performance between a source and a target dataset when those datasets have different language distributions. This bias can be expressed by an increase in perplexity, that is:

$$PPL_{LM}(\mathcal{T}) \gg PPL_{LM}(\mathcal{S}) \qquad (1)$$

where $PPL_{LM}(\mathcal{T})$ and $PPL_{LM}(\mathcal{S})$ are character-level perplexities of an external LM estimated on a source dataset $\mathcal{S}$ and evaluated on a target $\mathcal{T}$ or a source dataset $\mathcal{S}$, respectively. Such a language shift can appear regardless of the visual uniformity between the images in the source and target datasets, for instance when the same writer is asked to fill several fields on a personal data form. In this case, the writing can be visually very similar while the language varies a lot between name, surname and locality, for instance. In this regard, we henceforth name *subsets* two sets of images which are visually similar as they originate from the same dataset but differ in their language distributions.

We argue that by learning to model the language of the source subset corpus at training time, an NN becomes biased towards it. While this bias can provide satisfying performance on a source test set, it prevents decent recognition accuracy on a target test set whose language is shifted. Yet, we hypothesize that character features learnt by a vision-encoder are robust enough even if the encoder is trained on source images only. Hence, we focus on the decoder which learns to model language *via* auto-regression. Our goal is to reduce the influence of its implicit modeling of source language so that it instead relies on an external LM which we can exchange at inference time. We aim for a system which correctly reads scans from a shifted target subset *without training it on target image-text pairs*, instead leveraging *unpaired* text data.

## 4 n-gram injection for dynamic language model adaptation

In the present section, we first introduce our n-gram injection framework as a means to alleviate the performance drop on target data. This framework can complement any character-level encoder-decoder whose decoder is transformer-based. Second, we present the lightweight transformer-based encoder-decoder into which we inject the n-gram.

### 4.1 n-gram injection

**Method** Let us now introduce our early injection mechanism of a character-level n-gram into an auto-regressive transformer decoder, as illustrated in Fig. 1. Such
<figure><img src="image_2.png" /><figcaption>Fig. 1. n-gram injection (NGI) into the auto-regressive decoder of the word attention network (WAN). An n-gram distribution vector $S^{NGI}$ is made noisy by function $\phi$, projected by $f$ then summed with standard embeddings of previously output characters $\xi(c)$ and positional encoding $P$ to form the new decoder input $X$. Switching to an appropriate n-gram at test time reduces the error gap on target corpora.</figcaption></figure>

decoders commonly predict the next token in a sequence based on the embeddings and positional encoding of previously predicted tokens. We complement this standard framework by adding language information from an n-gram into the input of the decoder.

Starting from log-probabilities and back-off weights stored in ARPA format, n-gram scores of all characters given a context are computed then inverse-log transformed and grouped into an n-gram distribution vector denoted by $s_t^{NGI}$. At decoding step $t$, this is expressed by the following equation:

$$s_{t,k} = g([c_{t-(n-1)}, \dots, c_{t-1}]^\top) = \mathbb{P}(c_{t,k} | c_{t-(n-1)}, \dots, c_{t-1}) \\ s_t^{NGI} = [s_{t,1}, \dots, s_{t,K}]^\top \qquad (2)$$

where $c_{t,k}$ is a character relating to position $t$ in the sequence with $k$ the index of a character among character set $K$, and $g$ is a function which maps a context vector $[c_{t-(n-1)}, \dots, c_{t-1}]^\top$ to n-gram probabilities for each character based on log-probabilities and back-off weights read in the ARPA file, assuming that $t - (n-1) \ge 0$, otherwise the context is bounded by $c_0$ and $c_{t-1}$.

Following, to avoid over-fitting on source n-gram vectors, we add noise to them. This brings variability to the training process to let the network adapt to unseen target n-gram vectors later at inference. Let $\epsilon_t$ be a white noise:

$$\epsilon_t = [\epsilon_{t,1}, \dots, \epsilon_{t,K}]^\top, \epsilon_{t,k} \sim \mathcal{U}(a, b) \qquad (3)$$

This noise is applied at step $t$ with sample-wise probability $\tau$:

$$\phi(s_t^{NGI}) = \begin{cases} \psi(s_t^{NGI} + \epsilon_t), & \text{if } X < \tau, X \sim \mathcal{U}(0, 1) \\ s_t^{NGI}, & \text{otherwise} \end{cases} \qquad (4)$$
where $\phi: \mathbb{R}^K \to \mathbb{R}^K$ is the function applying noise with probability $\tau$, and
$\psi: \mathbb{R}^K \to \mathbb{R}^K$ normalizes back to a valid probability distribution after alteration by the noise $\epsilon_t$.

Finally, at step $t$, n-gram vectors resulting from Equation 4 from all previous steps are concatenated into a matrix $\mathbf{S}^{\text{NGI}} = [\mathbf{s}_1^{\text{NGI}}, \dots, \mathbf{s}_t^{\text{NGI}}]^{\top}$. This matrix $\mathbf{S}^{\text{NGI}}$ is given as input to a feed-forward projection then summed with the embeddings of output characters from previous steps $\xi(\mathbf{c})$ and positional encoding $\mathbf{P}$. We formalize the injection into transformer input $\mathbf{X}$ at decoding step $t$ as:

$$ \mathbf{X} = f(\phi(\mathbf{S}^{\text{NGI}})) + \xi(\mathbf{c}) + \mathbf{P} \quad (5) $$

where $\phi: \mathbb{R}^{t \times K} \to \mathbb{R}^{t \times K}$ is the same noise function as before but in the matrix domain, $f: \mathbb{R}^{t \times K} \to \mathbb{R}^{t \times d}$ is a feed-forward projection, $\mathbf{c} = [c_0, \dots, c_{t-1}]^{\top}$ is the concatenation of previous characters, $\xi: \mathbb{R}^t \to \mathbb{R}^{t \times d}$ and $\mathbf{P} \in \mathbb{R}^{t \times d}$ refer to an embedding function and positional encoding as in the original transformer [40], respectively, and $d$ is the internal dimension of the transformer decoder.

**Discussion** We choose n-gram models as external LM because they exhibit good properties for providing a neural architecture with an LM at a low computational cost. They are indeed lightweight and condensed representations of the language from a given corpus. Also, they can be estimated quickly as n-gram counting can be done in a couple minutes on a standard CPU. As opposed to sampling one discrete character (e.g. the one with maximum probability) from the n-gram, we argue that preserving the empirical distribution across all characters provides the NN with richer information. Besides, this framework is independent from the order $n$ of the n-gram, meaning that any $n$ can be used.

Injecting the LM into the input of the auto-regressive transformer decoder brings several benefits.

First, if a network models language on source data as is the case for standard transformers in the literature, then it keeps obeying the source language and make errors once evaluated on a different target corpus. Teaching the network to make use of a provided external LM instead allows to switch it seamlessly at inference time, without any modification to the network's parameters—not even of the projection $f$. This also means that it does not require any extra training on target image-text pairs, meaning we can leverage unpaired text data. By keeping the standard auto-regressive re-injection of previously predicted characters into the transformer decoder along with NGI, we allow the network to still learn some share of biased language rules during training, e.g. the English or French syntax, which remain relevant on the target subset.

Second, by injecting the LM at the very beginning of the decoder, we allow the transformer to fully integrate and benefit from the language information. Importantly, there is no hyper-parameter to manually tune since the network learns to control the appropriate importance to give to the n-gram LM, balancing it dynamically with encoder features to resolve visual ambiguities. By moving away from post-processing, we avoid re-scoring computations which otherwise
induce a sensible time complexity overhead during inference for they explore
a large search graph or lattice. Moreover, post-processing does not let the NN
adapt to the *n*-gram during training.

Third, setup and inference are low-cost. The overhead caused by NGI is
minimal since *n*-gram vectors have a low inference cost and can even be stored
in cache to improve memory reads. The parameters and computing cost of the
additional feed-forward projection *f* is also negligible compared to the rest of
the network which we leave as is.

## 4.2 Word attention network

We now introduce the word attention network (WAN), an FCN-transformer built
as a smaller counterpart of the well-known, full-page HTR model DAN [7]. With
2.1M parameters, the WAN is ten times smaller. We note that WAN is merely
one example of an architecture into which NGI can be plugged. A lightweight
architecture is more relevant in our setting as our training corpus is not suf-
ficiently large to train bigger models. Also, the handwritten word recognition
task is easier than full page HTR. We choose an FCN encoder because FCN
are efficient for image feature extraction and can process variable input shapes,
which is suited for scanned text images. For the decoder part, we opt for a trans-
former since such attention-based models are commonly used in state-of-the-art
approaches in HTR. Full architectural details are given in Section 6.2.

# 5 Datasets

As shown in Table 1, we conduct experiments on a total of five handwritten word-
level dataset splits, four of which are custom splits of the well-known IAM [24]
and RIMES [14] datasets while the latter is an industrial use case on name and
surname recognition in forms. The custom splits are provided to exacerbate a
language shift and its associated bias, as discussed later on. Additionally, we
report the 5-gram character-level perplexity (PPL) on test sets to assess the
strength of the shift between source and target language distributions, where a
higher PPL denotes a bigger shift. Third to last and second to last columns use
a 5-gram estimated on the source development set, i.e. the union of the training
and validation sets. Last column uses a 5-gram estimated on the target *n*-gram
set, denoted by NG. This target NG set was created to estimate our *n*-grams,
ensuring that they are not computed directly on target test sets.

5.1 Perplexity-controlled splits on IAM & RIMES

The IAM [24] dataset is a collection of modern documents in English. Word
text-image pairs are from the Aachen split, also known as IAM-A. Regarding the
RIMES dataset, we use the 2011 version [14] which consists of French adminis-
trative letters. Word images are extracted from the body of the letters, excluding
addresses and signatures. We provide custom splits of IAM and RIMES as official
<table><caption>**Table 1. Size and perplexity of the five word-level HTR dataset splits.** Dev. denotes the source development set, NG the target n-gram set.</caption><thead><tr><th rowspan="3">Dataset</th><th rowspan="3">Split</th><th colspan="4">Size</th><th colspan="3">5-gram PPL</th></tr><tr><th colspan="2">Source</th><th colspan="2">Target</th><th colspan="2">Source dev.</th><th>Target NG</th></tr><tr><th>Dev.</th><th>Test</th><th>NG</th><th>Test</th><th>Source</th><th>Target</th><th>Target</th></tr></thead><tbody><tr><td rowspan="2">IAM</td><td>Lexicon</td><td>69,595</td><td>6,507</td><td>6,511</td><td>7,187</td><td>4.6</td><td>14.5</td><td>4.3</td></tr><tr><td>k-means</td><td>69,595</td><td>6,730</td><td>6,511</td><td>6,964</td><td>4.6</td><td>26.2</td><td>3.8</td></tr><tr><td rowspan="2">RIMES</td><td>Lexicon</td><td>51,908</td><td>5,022</td><td>4,857</td><td>5,191</td><td>3.1</td><td>8.3</td><td>2.4</td></tr><tr><td>k-means</td><td>51,908</td><td>5,670</td><td>4,857</td><td>4,543</td><td>3.1</td><td>66.4</td><td>2.4</td></tr><tr><td>N2S (private)</td><td>—</td><td>119,735</td><td>9,101</td><td>879,350</td><td>9,635</td><td>3.5</td><td>24.5</td><td>18.9</td></tr></tbody></table>

splits do not naturally exhibit a language shift. We consider two splitting strategies, referred to as *lexicon* and *k-means* splits. As reported in Table 1, source training, validation and test set respectively contain 70%, 7.5% and 7.5% of the samples. The remaining 15% percent are equally shared between the target NG and test sets. All splits are publicly released<fnref n="1" />.

**Lexicon split** A way to ensure a significant shift between source and target languages is to divide the lexicon $\mathcal{L}$ of the whole dataset into two, that is one lexicon for the source subset $\mathcal{L}_S$ and another for the target subset $\mathcal{L}_T$, such that $\mathcal{L}_S \cap \mathcal{L}_T = \emptyset$.

**k-means split** Another means of creating a shift between languages is to embed words into a latent space in which we separate the projections into two clusters. We represent each word $w$ of the lexicon $\mathcal{L}$ by a boolean vector $v_w$ where $v_{w,i}=1$ if the 2-gram $i$ is present in $w$, so that the size of $v_w$ equals the number of 2-grams in the corpus. Then, we split those vectors into two clusters using the k-means algorithm with $k=2$. Here, $\mathcal{L}_S \cap \mathcal{L}_T \neq \emptyset$, yet the source-development-to-target-test PPL is higher than that of the lexicon split. It is expected that the source-to-target performance gap will be larger on the k-means split than on the lexicon split. Indeed, we assume a higher source-to-target PPL expresses a more pronounced language bias and thus presumes a more challenging recognition.

## 5.2 N2S private dataset

The name to surname private dataset, denoted by N2S, is a real-world scenario of name and surname recognition in handwritten complaint forms, four samples of which are shown in Fig. 2. In this industrial dataset, the name and surname of complaining users must be read. For this purpose, NG is an external database containing a census of surnames with their number of occurrences.

<footnote n="1">¹ https://www.kaggle.com/datasets/papericdar2026/iam-and-rimes2011-custom-splits.</footnote>
<figure><img src="image_3.png" /><figcaption>Fig. 2. Scans of names (left) and surnames (right) from the N2S dataset.</figcaption></figure>

# 6 Experiments

## 6.1 Evaluation protocol

In our experiments, we compare the proposed WAN to three encoder-decoders to estimate the state-of-the-art transfer performance of biased architectures: TrOCR [22], a well-known transformer-only architecture for text line recognition; the DAN [7], a full-page FCN-transformer HTR model; and SaLT [25], a debiased and lightweight FCN-transformer designed to recognize arbitrary characters sequences as on vehicle license plates. All three rely on auto-regressive character-level decoders. We report the standard character error rate (CER) metric to evaluate system performance, expressed as a percentage. Regarding NGI LM switching, we use an *n*-gram estimated on the source development set when evaluating on the source test set, whereas evaluation on the target test set uses an *n*-gram estimated on the target *n*-gram set NG (different from the target test set).

## 6.2 Training details

All models are trained until convergence for 100 epochs on the source training subset only. They are all trained from scratch except TrOCR whose encoder uses pre-trained weights from the first-stage checkpoint<fnref n="2" /> since it could not achieve satisfying results on such small datasets otherwise. Notably, transformer decoders benefit from training from scratch because pre-training would only bias them towards the source training corpus [32]. Images keep their original shapes and follow ImageNet normalization for all models except TrOCR which keeps its custom image resizing and pre-processing strategy. Heavy image augmentation is performed in the same way as Coquenet et al. [7] with DPI scaling, perspective change, elastic distortion, erosion, color jittering, Gaussian blur and noise, sharpening and random zooming. There is a 0.9 probability that these augmentations are applied, with a 0.2 probability for each of them. No synthetic data is used. Optimization is done on the cross-entropy loss with the Adam optimizer [19]. Model selection is performed according to validation CER on the source subset.

We use TrOCR<sub>SMALL</sub> variant throughout this paper. For a fairer comparison, we replace TrOCR's byte-pair-encoding (BPE) [30] tokenizer with a character-level one. We then fine-tune the whole TrOCR, with a batch size of 256 and a learning rate of 2 × 10<sup>-5</sup>. DAN and SaLT are trained with a batch size of 128 and a learning rate of 1 × 10<sup>-4</sup> and 3 × 10<sup>-4</sup>, respectively.

<footnote n="2">² https://huggingface.co/microsoft/trocr-small-stage1.</footnote>
WAN consists in a ten-layer FCN encoder and a small two-layer transformer
decoder, down from the 6 and 8 decoder layers of TrOCR and the DAN. Each
of the ten layers comprises a convolution, ReLU activation, instance normaliza-
tion [39], 2 × 2 max pooling and dropout [33] with a probability of 0.1. Each
layer’s convolution has a 3 × 3 kernel, a stride of 1 and padding in such a way
that the output has the same shape as the input. Output channels are increased
so that the first three layers output 32 channels, the next three 64, the follow-
ing three 128 and the final layer 256. The transformer decoder has an internal
dimension *d* = 256 with 8 heads. We employ teacher forcing error (TFE) with
probability 0.1 in the WAN’s transformer decoder during training to simulate
wrong previous predictions which might occur at inference time. This demon-
strated better generalization abilities in HTR [7]. TFE also impacts NGI, as
changing the context generates alternative *n*-gram vectors. Through simulat-
ing erroneous predictions, TFE restrains the network’s reliance on both internal
language modeling and external LM by reducing its confidence into previous
predictions. The WAN is trained with a batch size of 128 and a learning rate of
3 × 10⁻⁴.

Regarding NGI, all *n*-grams referred to in this paper are character-level and
use interpolation with Witten-Bell smoothing [43]. We use a value of *n* = 5
unless stated otherwise, as it is a good trade-off between estimation quality and
compute time. The SRILM toolkit is used to estimate *n*-grams. White noise *ε* uses
*a* = -0.1, *b* = 0.1, *τ* = 0.2 following hyper-parameter search on the source subset.
The projection *f* is a feed-forward layer followed by layer normalization [1] and
ReLU activation.

6.3 Performance drop of biased state-of-the-art networks

Tables 2 and 3 present results on source and target subsets without NGI. All three state-of-the-art models and the WAN suffer from a significant performance gap between source and target subsets of IAM, RIMES and N2S. When comparing splits in each dataset, we observe that source CER are almost equal while CER approximately double from lexicon to *k*-means target splits, except for SaLT whose debiasing reduces the gap as discussed later on. This doubling in CER is linked to the surge in PPL from source to target that is higher among *k*-means splits than lexicon splits, as reported in Table 1: the higher the difference in PPL between subsets, the larger the gap in CER. In particular, the WAN without NGI suffers from a performance gap similar to state-of-the-art alternatives, rising from 7.6 to 23.4 CER and from 4.3 to 29.9 CER on *k*-means splits of IAM and RIMES, respectively. Likewise for N2S, the CER more than doubles from source to target test set. As expected, the WAN without NGI is no exception with a CER increasing from 2.5 to 6.3. Such poor error rates make a transcription system unusable in practice, which we aim to alleviate thanks to NGI as described in the next section.

Although TrOCR and the DAN were proposed for line-level and page-level
HTR, respectively, we observe that they effortlessly adapt to word-level HTR
on source test sets. However, they suffer from a drop comparable to that of the
<table><caption>**Table 2.** CER(%) of transformer-based models trained on the four custom dataset splits from IAM and RIMES. All four models suffer from a significant performance gap between source and target subsets, including the WAN without NGI.</caption><thead><tr><th rowspan="3">Model</th><th colspan="4">IAM</th><th colspan="4">RIMES</th></tr><tr><th colspan="2">Lexicon split</th><th colspan="2">k-means split</th><th colspan="2">Lexicon split</th><th colspan="2">k-means split</th></tr><tr><th>Source</th><th>Target</th><th>Source</th><th>Target</th><th>Source</th><th>Target</th><th>Source</th><th>Target</th></tr></thead><tbody><tr><td>TrOCR [22]</td><td>6.3</td><td>12.9</td><td>6.7</td><td>21.4</td><td>3.3</td><td>12.9</td><td>3.6</td><td>33.3</td></tr><tr><td>DAN [7]</td><td>6.0</td><td>15.0</td><td>6.1</td><td>26.1</td><td>3.7</td><td>16.8</td><td>3.9</td><td>40.2</td></tr><tr><td>SaLT [25]</td><td>15.5</td><td>25.1</td><td>15.5</td><td>29.8</td><td>10.7</td><td>25.9</td><td>12.3</td><td>38.6</td></tr><tr><td>WAN<sup>3</sup></td><td>6.9</td><td>15.8</td><td>7.6</td><td>23.4</td><td>4.7</td><td>18.1</td><td>4.3</td><td>29.9</td></tr></tbody></table>

<table><caption>**Table 3.** CER(%) of transformer-based models trained on the name to sur-name (N2S) dataset. As on IAM and RIMES, all four models suffer from a performance gap between source and target corpora, including the WAN without NGI.</caption><thead><tr><th>Model</th><th>Source</th><th>Target</th></tr></thead><tbody><tr><td>TrOCR [22]</td><td>2.2</td><td>7.1</td></tr><tr><td>DAN [7]</td><td>2.1</td><td>5.3</td></tr><tr><td>SaLT [25]</td><td>4.0</td><td>13.4</td></tr><tr><td>WAN<sup>3</sup></td><td>2.5</td><td>6.3</td></tr></tbody></table>

WAN on target test sets, which is expected as they all absorb training corpus bias. Note that TrOCR and the DAN are not evaluated with off-the-shelf pre-trained weights as this would not result in a fair comparison with SaLT and the WAN which must be trained from scratch. The comparatively poor performance of SaLT can be explained by its debiased design which stops syntactical modeling of the training language. This proved to be successful on vehicle license plates which do not follow any rules in the ordering of their characters, however in handwriting recognition, learning language-specific rules is crucial for improving prediction performance.

## 6.4 Reducing the performance gap

We now investigate the positive impact on performance retention provided by NGI into the WAN. Table 4 shows that NGI ensures a lower target CER on all four custom splits of IAM and RIMES while preserving source CER. The WAN+NGI performs better on target corpora than the WAN alone and than state-of-the-art models from Table 2. Indeed, the WAN+NGI reaches CER of 11.8 and 10.1 on the lexicon and k-means target test sets for IAM, respectively. On RIMES target test sets, it obtains a CER of 8.8 on the lexicon split and 19.2 on the k-means split. A similar trend is reported on N2S in Table 5 where the

<footnote n="3">³ For a fair comparison, here the WAN is trained without TFE to match the training strategy of other models.</footnote>
<table><caption>**Table 4. CER(%) of the proposed NGI compared to the WAN alone on the four custom dataset splits from IAM and RIMES.** NGI consistently reduces the performance gap from source to target subsets. A post-processing LM, denoted by +LM, further reduces the gap. We highlight in bold the best results on target test sets as they are our main objective.</caption><thead><tr><th rowspan="3">Model</th><th colspan="4">IAM</th><th colspan="4">RIMES</th></tr><tr><th colspan="2">Lexicon split</th><th colspan="2">k-means split</th><th colspan="2">Lexicon split</th><th colspan="2">k-means split</th></tr><tr><th colspan="2">Source Target</th><th colspan="2">Source Target</th><th colspan="2">Source Target</th><th colspan="2">Source Target</th></tr></thead><tbody><tr><td>WAN</td><td>6.0</td><td>13.2</td><td>6.8</td><td>18.0</td><td>3.4</td><td>12.2</td><td>4.0</td><td>25.4</td></tr><tr><td>WAN+NGI</td><td>6.2</td><td><strong>11.8</strong></td><td>6.8</td><td><strong>10.1</strong></td><td>3.5</td><td><strong>8.8</strong></td><td>3.6</td><td><strong>19.2</strong></td></tr><tr><td>WAN+LM</td><td>5.9</td><td><strong>10.6</strong></td><td>6.8</td><td>9.7</td><td>3.3</td><td>6.6</td><td>3.7</td><td>13.8</td></tr><tr><td>WAN+NGI+LM</td><td>6.5</td><td>11.5</td><td>6.9</td><td><strong>8.4</strong></td><td>3.4</td><td><strong>5.1</strong></td><td>3.7</td><td><strong>10.7</strong></td></tr></tbody></table>

<table><caption>**Table 5. CER(%) of the proposed NGI compared to the WAN alone on the name to surname (N2S) dataset.** As on IAM and RIMES, NGI reduces the drop on the target subset. We highlight in bold the best result on the target test set as it is our main objective.</caption><thead><tr><th>Model</th><th>Source</th><th>Target</th></tr></thead><tbody><tr><td>WAN</td><td>2.2</td><td>5.0</td></tr><tr><td>WAN+NGI</td><td>2.4</td><td><strong>4.5</strong></td></tr></tbody></table>

WAN+NGI obtains a target test CER of 4.5, proving that the proposed injection of a relevant n-gram helps reading out-of-distribution handwriting scans.

Moreover, NGI allows the network to better compensate the higher bias on k-means splits. Notably, on IAM, the WAN suffers from a source-to-target gap from 6.8 to 18.0 CER on the more challenging k-means split, which is higher than the increase from 6.0 to 13.2 CER on the lexicon split. On the contrary, the WAN+NGI has a smaller gap on the k-means split, from 6.8 to only 10.1 CER, than on the lexicon split which shows an increase from 6.2 to 11.8 CER. Nonetheless, the performance gain on the N2S dataset is minimal which can be justified by a much higher target PPL of 18.9 as displayed in Table 1. A quick analysis of the surnames census reveals that this database contains erroneous fields that do not correspond to surnames, for example numerical values.

An additional post-processing LM, denoted by +LM in Table 4, further improves target performance on IAM and RIMES. This LM has the same configuration as the one used for NGI. Lattice re-scoring is done using the SRILM toolkit with beam width equal to 150. Both the WAN and WAN+NGI benefit from applying an LM for post-processing. Note that using the n-gram as a post-processing (WAN+LM) is slightly better than injecting into the input of the network (WAN+NGI), at the cost of an increased computational complexity. Finally, the combination of the two n-gram methods into a WAN+NGI+LM attains the best performance on the target test set.
<table><caption>**Table 6.** Ablation study of our proposal (WAN+NGI) by removing specific components of NGI and reducing the *n*-gram order on the *k*-means splits from IAM and RIMES. We report CER(%) when removing noise or the TFE training strategy from NGI, and lowering *n* to {3, 2} for NGI. Ablations are not cumulative.</caption><thead><tr><th rowspan="2">Model</th><th colspan="2">IAM</th><th colspan="2">RIMES</th></tr><tr><th>Source</th><th>Target</th><th>Source</th><th>Target</th></tr></thead><tbody><tr><td>WAN+NGI (5-gram)</td><td>6.8</td><td>10.1</td><td>3.6</td><td>19.2</td></tr><tr><td>- Noise φ</td><td>6.6</td><td>10.6</td><td>3.7</td><td>19.2</td></tr><tr><td>- TFE</td><td>10.3</td><td>16.5</td><td>5.9</td><td>20.8</td></tr><tr><td>3-gram</td><td>6.8</td><td>18.1</td><td>3.2</td><td>26.2</td></tr><tr><td>2-gram</td><td>6.7</td><td>19.3</td><td>3.3</td><td>25.5</td></tr></tbody></table>

## 6.5 Ablation study of NGI components

In Table 6, we present results after separately ablating NGI components and changing the order *n* of the injected *n*-gram. We perform the ablation study on the *k*-means splits from IAM and RIMES as their higher perplexity makes them more challenging. Removing noise slightly degrades CER as the network tends to overfit on the training corpus. Notably, $s_1^{NGI}$ is computed from the start-of-sequence token as context, producing a constant vector $s_1^{NGI}$ for a given LM. Thus, adding noise to $s_1^{NGI}$ provides variability when learning to predict the first token, improving adaptability to new LM during inference. Removing TFE reduces generalization capabilities regarding both auto-regressively injected tokens and *n*-grams, altering CER on both target and source test sets. Switching to lower-order 3-grams or 2-grams causes the largest performance drop on target corpora. It is expected that the closer the *n*-gram estimate of the underlying target distribution, the smaller the performance drop. In these cases, network training is done with an NGI of the same order *n* as that injected at test time.

## 7 Conclusion & future work

This paper investigates the robustness of transformer-based encoder-decoders to language shift in word-level HTR. After revealing the recognition performance drop of state-of-the-art architectures when the test language distribution differs from training, we propose *n*-gram injection (NGI) as a method for dynamically injecting an external *n*-gram in order to reduce the error surge. In addition, we introduce the word attention network (WAN) as a lightweight FCN-transformer for handwritten word recognition, in which NGI is applied. Through experiments, NGI demonstrates successful character recognition under linguistic distribution shifts, complementing the transformer decoder of the WAN. By switching to an appropriate *n*-gram at test time, our approach mitigates bias without any extra training of the neural recognizer. Comparing results with lattice rescoring suggests that NGI is a promising step towards moving away from costly
post-processing. Lastly, we expect our framework to be compatible with neu-
ral external language models, which future work could for instance leverage for
challenging cross-lingual transfer if computational complexity is less of an issue.

**Acknowledgments.** This study was funded by the CIFRE ANRT grant No. 2023/0195 and contributes to AI Excellence Cluster SequoIA (grant ANR-23-IACL-0009).

References

1. Ba, J.L., Kiros, J.R., Hinton, G.E.: Layer Normalization (2016). https://doi.org/10.48550/arXiv.1607.06450

2. Barrere, K., Soullard, Y., Lemaitre, A., Coüasnon, B.: A Light Transformer-Based Architecture for Handwritten Text Recognition. In: Document Analysis Systems, vol. 13237, pp. 275–290. Springer International Publishing (2022)

3. Barrere, K., Soullard, Y., Lemaitre, A., Coüasnon, B.: Training transformer architectures on few annotated data: An application to historical handwritten text recognition. International Journal on Document Analysis and Recognition **27**, 553–566 (2024)

4. Bluche, T., Louradour, J., Knibbe, M., Moysset, B., Benzeghiba, M.F., Kermorvant, C.: The A2iA Arabic Handwritten Text Recognition System at the Open HaRT2013 Evaluation. In: 2014 11th IAPR International Workshop on Document Analysis Systems. pp. 161–165 (2014)

5. Coquenet, D.: Meta-dan: towards an efficient prediction strategy for page-level handwritten text recognition. Pattern Recognition (2026)

6. Coquenet, D., Chatelain, C., Paquet, T.: Recurrence-free unconstrained handwritten text recognition using gated fully convolutional network. In: 17th International Conference on Frontiers in Handwriting Recognition. pp. 19–24 (2020)

7. Coquenet, D., Chatelain, C., Paquet, T.: DAN: A Segmentation-free Document Attention Network for Handwritten Document Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence pp. 1–17 (2023)

8. Diao, S., Bai, J., Song, Y., Zhang, T., Wang, Y.: ZEN: Pre-training Chinese Text Encoder Enhanced by N-gram Representations. In: Findings of the Association for Computational Linguistics: EMNLP 2020. pp. 4729–4740 (2020)

9. Diao, S., Xu, R., Su, H., Jiang, Y., Song, Y., Zhang, T.: Taming Pre-trained Language Models with N-gram Representations for Low-Resource Domain Adaptation. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). pp. 3336–3349 (2021)

10. Garrido-Munoz, C., Calvo-Zaragoza, J.: On the Generalization of Handwritten Text Recognition Models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 15275–15286 (2025)

11. Graves, A., Fernández, S., Gomez, F., Schmidhuber, J.: Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. In: Proceedings of the 23rd International Conference on Machine Learning. pp. 369–376 (2006)

12. Graves, A., Liwicki, M., Fernández, S., Bertolami, R., Bunke, H., Schmidhuber, J.: A Novel Connectionist System for Unconstrained Handwriting Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence **31**(5), 855–868 (2009)
13. Graves, A., Schmidhuber, J.: Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks. In: Advances in Neural Information Processing Systems. vol. 21 (2008)

14. Grosicki, E., El-Abed, H.: ICDAR 2011 - French Handwriting Recognition Competition. In: 2011 International Conference on Document Analysis and Recognition. pp. 1459-1463 (2011)

15. Gulcehre, C., Firat, O., Xu, K., Cho, K., Barrault, L., Lin, H.C., Bougares, F., Schwenk, H., Bengio, Y.: On Using Monolingual Corpora in Neural Machine Translation (2015). https://doi.org/10.48550/arXiv.1503.03535

16. Hu, J., Liu, C., Yan, Q., Zhu, X., Wu, J., Du, J., Dai, L.: Vision-Language Adaptive Mutual Decoder for OOV-STR. In: Image and Graphics. pp. 175-186 (2023)

17. Jurafsky, D., Martin, J.: Speech and Language Processing. Pearson Education (2014)

18. Kang, L., Riba, P., Villegas, M., Fornés, A., Rusiñol, M.: Candidate fusion: Integrating language modelling into a sequence-to-sequence handwritten word recognition architecture. Pattern Recognition 112, 107790 (2021)

19. Kingma, D.P., Ba, J.: Adam: A Method for Stochastic Optimization (2017). https://doi.org/10.48550/arXiv.1412.6980

20. Leviathan, Y., Kalman, M., Matias, Y.: Fast Inference from Transformers via Speculative Decoding. In: Proceedings of the 40th International Conference on Machine Learning. pp. 19274-19286 (2023)

21. Li, H., Cai, D., Xu, J., Watanabe, T.: Residual Learning of Neural Text Generation with n-gram Language Model. In: Findings of the Association for Computational Linguistics: EMNLP 2022. pp. 1523-1533 (2022)

22. Li, M., Lv, T., Chen, J., Cui, L., Lu, Y., Florencio, D., Zhang, C., Li, Z., Wei, F.: TrOCR: Transformer-based optical character recognition with pre-trained models. In: Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence. vol. 37, pp. 13094-13102 (2023)

23. Liu, J., Min, S., Zettlemoyer, L., Choi, Y., Hajishirzi, H.: Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokens. In: First Conference on Language Modeling (2024)

24. Marti, U.V., Bunke, H.: The IAM-database: An English sentence database for offline handwriting recognition. International Journal on Document Analysis and Recognition 5(1), 39-46 (2002)

25. Meyer, F., Guichard, L., Coquenet, D., Gravier, G., Soullard, Y., Coüasnon, B.: Relaxed Syntax Modeling in Transformers for Future-Proof License Plate Recognition. In: Document Analysis and Recognition - ICDAR 2025. pp. 154-171 (2026)

26. Michael, J., Labahn, R., Grüning, T., Zöllner, J.: Evaluating Sequence-to-Sequence Models for Handwritten Text Recognition. In: 2019 International Conference on Document Analysis and Recognition. pp. 1286-1293 (2019)

27. Park, S., Chung, S., Lee, J., Choo, J.: Improving Scene Text Recognition for Character-Level Long-Tailed Distribution (2023). https://doi.org/10.48550/arXiv.2304.08592

28. Pippi, V., Cascianelli, S., Kermorvant, C., Cucchiara, R.: How to Choose Pre-trained Handwriting Recognition Models for Single Writer Fine-Tuning. In: Document Analysis and Recognition - ICDAR 2023. pp. 330-347 (2023)

29. Roy, A., Anil, R., Lai, G., Lee, B., Zhao, J., Zhang, S., Wang, S., Zhang, Y., Wu, S., Swavely, R., Tao, Yu, Dao, P., Fifty, C., Chen, Z.,
Wu, Y.: N-Grammer: Augmenting Transformers with latent n-grams (2022).
https://doi.org/10.48550/arXiv.2207.06366

30. Sennrich, R., Haddow, B., Birch, A.: Neural Machine Translation of Rare Words with Subword Units. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 1715–1725 (2016)

31. Soullard, Y., Swaileh, W., Tranouez, P., Paquet, T., Chatelain, C.: Improving Text Recognition using Optical and Language Model Writer Adaptation. In: 2019 International Conference on Document Analysis and Recognition. pp. 1175–1180 (2019)

32. Sriram, A., Jun, H., Satheesh, S., Coates, A.: Cold Fusion: Training Seq2Seq Models Together with Language Models. In: Proc. Interspeech 2018. pp. 387–391 (2018)

33. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research 15(56), 1929–1958 (2014)

34. Stewart, L., Trager, M., Gonugondla, S., Soatto, S.: The N-Grammys: Accelerating Autoregressive Inference with Learning-Free Batched Speculation. In: Proceedings of The 4th NeurIPS Efficient Natural Language and Speech Processing Workshop. pp. 322–335. PMLR (2024)

35. Swaileh, W., Paquet, T., Soullard, Y., Tranouez, P.: Handwriting Recognition with Multigrams. In: 2017 14th IAPR International Conference on Document Analysis and Recognition. vol. 01, pp. 137–142 (2017)

36. Tarride, S., Kermorvant, C.: Revisiting N-Gram Models: Their Impact in Modern Neural Networks for Handwritten Text Recognition. In: Document Analysis and Recognition - ICDAR 2024. pp. 167–182 (2024)

37. Tassopoulou, V., Retsinas, G., Maragos, P.: Enhancing Handwritten Text Recognition with N-gram sequence decomposition and Multitask Learning. In: 2020 25th International Conference on Pattern Recognition (ICPR). pp. 10555–10560 (2021)

38. Tian, J., Li, Z., Li, J., Wang, P.: N-gram Unsupervised Compoundation and Feature Injection for Better Symbolic Music Understanding. Proceedings of the AAAI Conference on Artificial Intelligence 38(14), 15364–15372 (2024)

39. Ulyanov, D., Vedaldi, A., Lempitsky, V.: Improved Texture Networks: Maximizing Quality and Diversity in Feed-Forward Stylization and Texture Synthesis. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 4105–4113 (2017)

40. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. pp. 6000–6010 (2017)

41. Voigtlaender, P., Doetsch, P., Ney, H.: Handwriting Recognition with Large Multi-dimensional Long Short-Term Memory Recurrent Neural Networks. In: 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR). pp. 228–233 (2016)

42. Wan, Z., Zhang, J., Zhang, L., Luo, J., Yao, C.: On Vocabulary Reliance in Scene Text Recognition. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 11422–11431 (2020)

43. Witten, I., Bell, T.: The zero-frequency problem: Estimating the probabilities of novel events in adaptive text compression. IEEE Transactions on Information Theory 37(4), 1085–1094 (1991)

44. Xiao, D., Li, Y.K., Zhang, H., Sun, Y., Tian, H., Wu, H., Wang, H.: ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. pp. 1702–1715 (2021)
45. Yousef, M., Hussain, K.F., Mohammed, U.S.: Accurate, data-efficient, unconstrained text recognition with convolutional neural networks. Pattern Recognition **108**, 107482 (2020)

46. Zhang, X., Zhu, B., Yao, X., Sun, Q., Li, R., Yu, B.: Context-Based Contrastive Learning for Scene Text Recognition. Proceedings of the AAAI Conference on Artificial Intelligence **36**(3), 3353–3361 (2022)

47. Zheng, T., Chen, Z., Huang, B., Zhang, W., Jiang, Y.G.: MRN: Multiplexed Routing Network for Incremental Multilingual Text Recognition. In: 2023 IEEE/CVF International Conference on Computer Vision (ICCV). pp. 18598–18607 (2023)