## 3 Fine-Grained NER Model
Then, \(C_{0,n+1}\) is a randomly-initialized character embedding of each character. A concatenation of three successive character embeddings (i.e., a character tri-gram embedding; \([C_{i-1};C_{i};C_{i+1}]\) is used as an input embedding \(Emb_{i}^{1}\) for the first stacked feature fusion layer. In the second stacked feature fusion (i.e., the middle layer) of Figure 1, \(POS_{C_{i}}\) is a character-unit POS tag of the \(i\)-th input character according to a BIO-tagging scheme similar to Table 1. Then, \(Emb(POS_{C_{i}})\) is a randomly-initialized POS embedding of the \(i\)-th character. To enrich input characters with grammatical information, a POS tri-gram embedding, \([Emb(POS_{C_{i-1}});Emb(POS_{C_{i}});Emb(POS_{C_{i+1}})]\), is concatenated with the tri-gram character embedding, \(Emb_{i}^{1}\), by using residual connections. The concatenated embedding is used as the input embedding \(Emb_{i}^{2}\) for the second stacked feature fusion layer. In the last stacked feature fusion layer (i.e., the uppermost layer) of Figure 1, \(DIC_{C_{i-1},C_{i};C_{i+1}}\) is a dictionary look-up feature on whether a character tri-gram, \([C_{i-1};C_{i};C_{i+1}]\), exists in a dictionary including character trigrams of predefined NE lists (i.e., NE lists in a training data). Then, \(Emb(DIC_{C_{i-1},C_{i};C_{i+1}})\) is a randomly-initialized dictionary embedding of the i-th character. To enrich input characters with domain knowledge, the dictionary embedding is concatenated with the character trigram embedding, \(Emb_{i}^{1}\), by using residual connections. The concatenated embedding is used as the input embedding \(Emb_{i}^{3}\) for the last stacked feature fusion layer. The hidden states of each stacked layer are concatenated as \(\overleftrightarrow{H}=[\overleftrightarrow{H}1;\overleftrightarrow{H}2; \overleftrightarrow{H}3]\). The lower output layer shown in Figure 1, calculates the degrees of association between H and the coarse-grained NE tag embeddings \(Emb(NE^{c})=\{Emb(NE^{c}_{1}),\;Emb(NE^{c}_{2}),\;\ldots,\;Emb(NE^{c}_{m})\}\) based on a multi-head attention mechanism [17], as shown in the following equation. \[head_{j} = attention(QW^{Q}_{j},KW^{K}_{j},VW^{K}_{j})=\alpha_{j}*VW^{V}_{j},\] \[WhereQ = \overleftrightarrow{H},K=V=Emb(NE^{c}),\] \[\alpha_{j} = softmax(\frac{QW^{Q}_{j}*(KW^{k}_{j})^{T}}{\sqrt{d_{h}}}), \tag{5}\] \[A(c^{c}_{i}) = head_{1}\oplus head_{2}\oplus...\oplus head_{k}\]  
where \(W^{Q}_{j}\in\;R^{d_{h}\times\frac{d_{h}}{k}}\), \(W^{K}_{j}\in\;R^{d_{h}\times\frac{d_{h}}{k}}\), and \(W^{V}_{j}\in\;R^{d_{h}\times\frac{d_{h}}{k}}\) are the weighting parameters of the \(j\)-th parameter among \(k\) heads to be learned during training. Then, \(Emb(NE^{c})\) represent the embedding vectors of m coarse-grained NE tags that are randomly initialized and fine-tuned during training. The attention score \(\alpha_{j}\) is calculated using a scaled-dot product, where \(d_{h}\) is a dimension of \(H\) (same as the dimension of Coarse-grained NE embedding). The attention score vector \(A(C^{c}_{i})\) represents the degrees of associations between the contextualized input embedding \(\overleftrightarrow{h}_{i}\) of the \(i\)-th input character and each coarse-grained NE tag. In other words, the vector can be considered as a potential distribution of coarse-grained tags associated with an input character. In the prediction phase, the lower output layer returns coarse-grained NE tags, as shown in the following equation. \[\hat{E}^{c}_{i}=argmax(\hat{A}^{1}_{i},\hat{A}^{2}_{i},...,\hat{A}^{m}_{i}) \tag{6}\]  
where \(\hat{A}^{j}_{i}\) denotes the \(j\)-th one among m attention scores in the trained attention vector \(\hat{A}_{i}\). In the upper output layer in Figure 1, each coarse-grained attention score vector \(A(C^{c}_{i})\) is concatenated with the hidden states of each stacked layer \(H\) to enrich fine-grained NEs with the contextual information of coarse-grained NEs.