## 1 Introduction
To be specific, **(i)** Chinese character images are usually space discrete, with the smallest unit stroke, while the general images are continuous with unit pixel, so the random mask may not work; **(ii)** Chinese characters are human-generated signals that are information dense with highly structured semantics, and every stroke or radical part may have its special informative meaning; **(iii)** Each Chinese character can be represented as a time series of images that carry information throughout its fixed order of strokes, as shown in Figure 1.  
Driven by the above analysis, we propose a stroke-based autoencoder (SAE) to extract the morphological information from the sequences of each Chinese character images. Our framework consists of two parts: **(i)** the encoder network, which operates on the image patches of Chinese characters to obtain encoded patches, and **(ii)** the decoder network, which reconstructs the stroke sequence images of each corresponding Chinese character. Figure 1: **Our SAE architecture based on Vision Transformer. ** The encoder is applied on the image patches of the Chinese characters (we take ‘#§’ for example). The decoder reconstructs the stroke sequence images (the order of strokes) from the encoded patches. The encoder in our SAE pre-training model aims at extracting the structured information from stroke images, and the decoder learns to predict the sequences of the writing order of Chinese characters. Some example results are illustrated in Figure 2. We choose some Chinese characters, all of which have eight strokes, for a neat illustration. The left column (with a yellow background) is the images of Chinese characters 'B', 'B', 'B' and 'B' in training dataset, and the right column (with a pale blue background) are images of 'B', 'B', 'B', 'B' and 'B' in the validation data set. Each Chinese character can be divided into two radical components, as illustrated in the middle column (with a light green background). These characters are made up of seven basic radical components, as illustrated at the bottom of the figure. For each character, the first row is the original stroke sequence images, i.e., their canonical order of strokes, and the second row is the corresponding stroke sequence predicted by our SAE model. The example results demonstrate that our pre-training model can successfully predict and reconstruct the unseen strokes and radicals of Chinese characters. Therefore, our SAE fully learned the morphological information of Chinese characters, which can be applied to explore the application of Chinese character structures, such as zero-shot Chinese character recognition and the generation of stylish Chinese fonts. It can also be applied to enrich word embeddings for downstream NLP tasks such as Chinese Named Entity Recognition. In this paper, we focus on the morphological structures of Chinese characters and utilize our pre-training models to explore zero-shot Chinese character recognition and word embeddings. we design two different SAE architectures on different forms of stroke images for different tasks. To the best of our knowledge, we are the first one to systematically explore the stroke sequence restoration and prediction at the image level. The experimental results validate that our SAE outperforms the previous results [1; 12; 13] on zero-shot recognition of Chinese characters. What is more encouraging is that our SAE is pre-trained on a small and concise dataset with printed Chinese character images,  
Figure 2: Example results on training (seen class) and validation (unseen class) datasets. Each character has the original picture (above) and reconstructed picture (below). The left column is the Chinese character image in the training datasets, and the right column is the Chinese character image in the validation datasets. The four characters of each column consist of seven basic radical parts. but boosts huge performance in zero-shot recognition on a large and complicated dataset consisting of handwritten Chinese character images.