## 3 Proposed Method  
### Method Principle  
Our objective is to extract items such as name, phone number, and postal code recorded in the courier order images. We annotate these items in JSON format, referred to as the ground truth. We use JSON format, like [\"key\":\"value\"], to annotate the items to be extracted. The keys in this format will be transformed into special tokens and added to the vocabulary of the language decoder. The decoder outputs sequences containing these tokens, and we extract sequences with special tokens from the output. Among these, valid sequences are referred to as SEQ, and SEQ must adhere to the format \(<\)s_key\(>\)value\(<\)e_key\(>\) (\(<\)s_key\(>\) indicating the start of the sequence, and \(<\)e_key\(>\) indicating the end of the sequence). Any invalid sequence will be simply discarded. SEQ and the JSON format [\"key\":\"value\"] are one-to-one corresponding. For example, \(<\)s_city\(>\)Hang Zhou\(<\)e_city\(>\) corresponds to [\"city\":\"Hang Zhou\"]. We illustrate an example of the conversion from JSON to SEQ in Figure 2. During the training process, the cross-entropy loss between the generated sequences and the annotated sequences continuously decreases. Eventually, the model learns to generate the correct SEQ.  
Our model takes an image as input and outputs the text items we need to extract. However, the pixel resolution of text in images is much higher than that of words in text paragraphs, making it challenging for current language models to directly process such high-dimensional features. To address this issue, we employ a visual encoder to transform the high-dimensional images into feature vectors. Subsequently, we can use a SeqSeq language decoder to generate the required sequences, and finally, we convert the outputs of decoder into text in JSON format, completing the entire task. To obtain richer layout features and contextual information after the initial visual encoder, we introduce a feature enhancement module. The design of this module aims to introduce more semantic and structural information in the early stages of visual processing, contributing to the improvement of the accuracy of the final sequence generation.  
Specifically, we adopted the same expression as [10]. For the original RGB image \(x\in\mathbb{R}^{h\times w\times c}\), where \(h\) represents the height of the image, \(w\) represents the width of the image, and \(c\) represents the number of channels of the image. The visual encoder transforms it into an embedding \(\{e_{i}\mid e_{i}\in\mathbb{R}^{d},1\leq i\leq n\}\), where \(n\) is the number of feature vectors generated by the visual encoder, and \(d\) is the dimension of the feature vectors. This embedding \(\{e\}\) is then transformed through our feature enhancement module into another set of feature vectors \(\{y_{i}\mid y_{i}\in\mathbb{R}^{d},1\leq i\leq n\}\). This set of feature vectors \(\{y\}\) serves as the input to the language decoder. Given \(\{y\}\), the decoder will generate sequences \((q_{i})_{i=1}^{m}\), where \(q_{i}\in\mathbb{R}^{c}\) is the one-hot vector of the \(i\)-th token, \(v\) is the size of the token vocabulary, and \(m\) is a hyperparameter. Eventually, \(q_{i}\) will be transformed into a word \(w_{i}\) from the vocabulary of language decoder.