# Visual and Phonological Feature Enhanced Siamese BERT for Chinese Spelling Error Correction
## 3 Approach
### Glyph Graph  
#### 3.3.1 Shape Embeddings for Visual Features  
We retrieve shape embeddings for Chinese characters from a glyph graph. As illustrated in Figure 2, Chinese characters can be repeatedly disassembled until the structure can not be subdivided. For example, character \"\(\circled{\%}\)\" can be divided into \"\(\circled{\%}\)\" and \"\(\circled{\%}\)\", while \"\(\circled{\%}\)\" can be split into \" \(\circled{\%}\)\" and \"\(\circled{\%}\)\". The same is true for character \"\(\circled{\%}\)\". Although the shapes of Chinese characters are different from others, they share common basic components which are also Chinese characters. The structure information of characters are labeled by the Kanji Database Project ([[http://kanji-database.sourceforge.net/](http://kanji-database.sourceforge.net/)]([http://kanji-database.sourceforge.net/](http://kanji-database.sourceforge.net/)), accessed on 12 September 2021). The shape information of Chinese characters is represented by unicode standard--Ideographic Description Sequence (IDS). For example, character \"\(\circled{\%}\)\" is represented as \"U+60A8 \"\(\circled{\%}\)\", while character \"\(\circled{\%}\)\" is represented as \"U+4F60 \"\(\circled{\%}\)\". As illustrated in Figure 3, we build the glyph graph according to IDS labels of Chinese characters by connecting characters and their sub-components. Table 2 shows some statistics of the glyph graph. Figure 2: Chinese characters can be disassembled into smaller components, which are also Chinese characters. Considering that different character pairs have different similarity degrees and need to be distinguished, we make the glyph graph directed and weighted in terms of the number of strokes and the neighbors of graph nodes. Figure 4 shows an extreme situation that a frequently-used radical \" { } \" connects to 1758 characters. This indicates why the number of neighbors should be considered when setting up the edge weights. These characters sharing the same radical \" { } are similar, but their similarities should be different from each other. If every node is connected equally without weights, the characters connected by \" { } will be similar to each other with the same degree, which is not true. Besides, the \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Training Data** & **\# Line** & **Avg. Length** & **\# Errors** \\ \hline Auto Generated Data & 269,327 & 42.7 & 378,751 \\ SIGHAN 2013 & 350 & 49.2 & 350 \\ SIGHAN 2014 & 6526 & 49.7 & 10,087 \\ SIGHAN 2015 & 3174 & 30.0 & 4237 \\ Total & 279,377 & 42.7 & 393,425 \\ \hline
**Test Data** & **\# Line** & **Avg. Length** & **\# Errors** \\ \hline SIGHAN 2013 & 1000 (1000) & 74.1 & 1227 \\ SIGHAN 2014 & 1062 (526) & 50.1 & 782 \\ SIGHAN 2015 & 1100 (550) & 30.5 & 715 \\ \hline
**Graph** & & **\# Characters** & **\# Edges** \\ \hline Glyph Graph & & 80,627 & 169,745 \\ SpellGCN: Pronunciation Graph & 4753 & 112,687 \\ SpellGCN: Shape Graph & 4738 & 115,561 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Statistics information of the used data resources. The number in the bracket in #Line column denotes the number of sentences with errors. For instance, 1062(526) indicates that there are a total of 1062 sentences in the test set, where contains 526 sentences with wrong words. The test set contains correct sentences to assess whether the model is correct. Figure 3: The glyph graph is built by connecting Chinese characters with common components. number of character strokes is also important. Connected characters with closer stroke numbers may look more similar than others, such as character pair \"\(\big{\langle}\)\" and \"\(\big{\langle}\)\" are more similar compared to pair \" { \" and \"\(\big{\langle}\)\". Let \(G=\{V,E,W\}\) denote the graph where \(V=\{v_{0},v_{1},\ldots,v_{N}\}\) denote all Chinese characters, \(w_{ij}\in W\) denotes the weight of edge \(e_{ij}\in E\) pointing to character \(v_{j}\) from character \(v_{i}\).