UFCN et: Unsupervised Network based on F ourier transform and Convolutional attention for Oracle Character Recognition (2024)

Abstract: Oracle bone script (OBS) is the earliest writing system in China, which is of great value in the improvement of archaeology and Chinese cultural history. However, there are some problems such as the lack of labels and the difficulty to distinguish the glyphs from the background of OBS, which makes the automatic recognition of OBS in the real world not achieve the satisfactory effect. In this paper, we propose a character recognition method based on an unsupervised domain adaptive network (UFCNet). Firstly, a convolutional attention fusion module (CAFM) is designed in the encoder to obtain more global features through multi-layer feature fusion. Second, we construct a Fourier transform (FT) module that focuses on the differences between glyphs and backgrounds. Finally, to further improve the network’s ability to recognize character edges, we introduce a kernel norm-constrained loss function. Extensive experiments perform on the Oracle-241 dataset show that the proposed method is superior to other adaptive methods. The code will be available at https://github.com/zhouynan/UFCNet.

## introduction the oracle bone inscriptions (obis) mainly refer to the obis of yinxu, which are carved on tortoises in the shang dynasty. it is the earliest self-contained writing system in china,which is of great significance to the improvement of chinese cultural history and the study of the formation and evolution of chinese characters @xcite . the oracle bone character (obc) image of rubbings is mainly the original image obtained by experts on the unearthed tortoise shell, animal bone, and other text carriers. as the oracle bones have been buried underground for a long time, they are badly damaged or contaminated, and there is serious noise @xcite , which makes it very challenging to recognize obcs. early research methods mainly combine graph theory and topological properties. @xcite proposed an obis recognition method based on graph theory. they abstracted oracle into an undirected graph composed of only points and lines, and extracted its topological features. @xcite introduced the information of the adjacent points of the endpoint, and improved the recognition accuracy through the continuous recognition of multi-level feature coding. however, these methods cannot meet the real-world oracle recognition, which requires a lot of manpower and time. to help with the excavation of new oracle bones and the identification of unseen characters, the advent of deep neural networks has a great impact on the recognition of oracle bone character (obc) images. @xcite used cnns to map character images into euclidean space for classification by nearest neighbor rules. @xcite utilized a low-level representation associated with gabor and an intermediate representation associated with a sparse encoder and combines it with a cnn-based model. however, training a depth model requires a large number of labeled samples. @xcite proposed an unsupervised structured texture separation network (stsn) for oracle identification and a dataset of 241 classes of oracle-241 @xcite for unsupervised identification. they took handprint characters transcribed by experts with high resolution and clean backgrounds as source domains. accordingly, the original oracle character (scanned image) is taken as the target domains. they have achieved good results by finding a domain invariant feature space to align the distribution between two domains. in this paper, we propose a network (ufcnet) combining fourier transform and convolutional attention for oracle character recognition. the convolution attention fusion module (cafm) combines deep and shallow features to obtain more global information and a better position location of char-acters. additionally, we further design the fourier transform (ft) module that converts the image from the spatial domain to the frequency domain, aiming to capture the edge details of the glyphs more efficiently and provide rich functionality for the cafm. we utilize the ft module to capture the high-frequency information of character images and extract rich edge information. we also introduce a kernel norm-constrained loss function to improve the network's discriminative performance on edges. we conduct extensive experiments on the oracle-241 dataset, and the results demonstrate that our network exhibits better recognition performance in the realm of unsupervised adaptation. our main contributions are summarized as follows: to validate the effectiveness of our method, we conduct extensive experiments on the oracle-241 dataset and results demonstrate that ufcnet has better classification accuracy than the existing state-of-the-art (sota) unsupervised obis recognition method stsn. ## methods the ufcnet network proposed in this paper is shown in figure 1 . it adopts the unsupervised idea of stsn to transfer the knowledge of handprint oracle character recognition to scanned dataset. it consists of three encoders, one of which is a glyph-sharing encoder e n for extracting both handprint and scanned characters. it is a resnet-18 pre-trained on the imagenet dataset as a structural encoder. the other two are independent encoders e p used to extract the background features of handprint and scanned characters. specifically, e p consists of one convolution unit with a kernel size of 7x7 (convolution, batchnormalization, and relu) and four convolution units with a kernel size of 3, cafm and ft. the cafm can cascade the high-level and low-level features of handprint and scanned images to obtain rich global features. the ft module can capture more edge features of characters by using the advantage of converting the image to the frequency domain. d f , two image-level discriminators {d is , d it } and a classifier that is finally used to classify the recognized scanned characters. for the discriminators of images and features, the discriminative network structure uses in this paper is detailed in ## conclusion in this paper, we propose a new network ufcnet for the recognition of oracle character images. different from the recognition method of obcs based on cnns, we use the fourier transform to transfer the recognition of oracle character images from the image domain to the frequency domain and extract rich edge information. at the same time, we use the convolutional attention fusion module to fuse shallow features with deep features in multiple layers, which makes up for the important detailed features lost in the sampling process of the cnn. a large number of experiments show that our ufcnet has better recognition accuracy compared with sota methods. however, due to the serious incompleteness and blurring of obcs, our network still needs to be further improved in recognition.