# Delayed Combination of Feature Embedding in Bidirectional LSTM CRF for NER
## 5 Results and Discussion
### The Model Equipped with Our Dictionary Representation
However, our CNN-based dictionary representation could predict them correctly. Commonly on the two datasets, we can notice that the dictionary without BI tags shows a better F1 score. We think this result might be caused by the following two reasons: tag mismatches and many ambiguous words. First, the 'tag mismatches' are very often found in between the dictionary with BI tags and the CoNLL 2003 dataset, especially for Person. We may think that a given name of a person is generally classified with a begin tag, whereas a family name is typically assigned with an inside tag. However, in the dataset of real text, it is very common to refer to a person by only his/her family name without giving a given name. So, family names are assigned both begin and inside tags in the dataset. The most collected names were in the full form of names and the dictionary made from them often caused such tag mismatches and, as a result, degraded the performance. Secondly, the 'ambiguous words' are very often found in the dictionary with BI tags for OntoNotes 5.0. For example, 'Hong' is a word found in various named entities, such as FACILITY: 'Hong Kong International Airport', GPE: 'Hong Kong', EVENT: 'Hong Kong Jewish Film Festival', WORK_OF_ART: 'Hong Kong Garden', PERSON: '[PERSON]', '[PERSON]', LOCATION: 'Disney Hong Kong'. When building the dictionary with BI tags up, 'Hong' is classified with B-FACILITY, B-GPE, B-EVENT, \begin{table}
\begin{tabular}{l c c} \hline \hline
**OntoNotes 5.0** & **Begin-Tag** & **Inside-Tag** \\ \hline Person & 61.59\% & 78.85\% \\ Norp & 15.35\% & 19.64\% \\ Facility & 69.87\% & 76.72\% \\ Organization & 76.32\% & 86.32\% \\ Product & 23.14\% & 48.39\% \\ Event & 42.77\% & 47.17\% \\ Work of art & 52.36\% & 71.92\% \\ Law & 28.00\% & 54.72\% \\ Location & 75.57\% & 48.59\% \\ GPE & 70.09\% & 77.20\% \\ Language & 46.65\% & 0.00 \% \\ \hline \hline \end{tabular}
\end{table} 
Table 7: The coverage of the dictionary on the OntoNotes5.0 dataset. \begin{table}
\begin{tabular}{l c c c c} \hline \hline  & & & \multicolumn{2}{c}{**Dictionary**} \\
**Model** & **Dataset** & **Without Dic** & **With BI Tags** & **Without BI Tags** \\ \hline Delayed-BiLSTM-CRF (FT + LSTM + DIC) & CoNLL 2003 & 90.79 & 91.13 & **91.24** \\ Delayed-BiLSTM-CRF (FT + CNN + DIC) & OntoNote 5.0 & 87.87 & 87.57 & **88.07** \\ \hline \hline \end{tabular}
\end{table} 
Table 8: The comparison of our models with and without dictionary representation. B-WORK_OF_ART, B-PERSON, I-PERSON, and I-LOCATION tags. This information is used to create a dictionary representation of the word 'Hong'. Like this, many ambiguous words are incorrectly encoded into dictionary representations, among which some adjacent dictionary representations may form unseen patterns that never occurred in the training set but that could be found in the test set. As a result, those representations may pose a problem in that they drastically reduce the performance. This result is similar to that in the earlier work [44]. We also compared our model to the previous similar dictionary-enabled works to show the feasibility of our model. The comparison is given in Table 9. On the CoNLL 2003, our model could achieve a better F1 score than that of [PERSON] et al., 2015 and [PERSON] et al., 2018 but was worse than that of [PERSON] and [PERSON], 2016 and [PERSON] and [PERSON], 2018. However, [PERSON] and [PERSON], 2016 trained their model by merging the validation data with the training data and this could lead to improve the F1 score. [PERSON] and [PERSON], 2018 used the lexical similarity (LS) embedding which was pre-trained from fined-grained named entity list and Wikipedia text. The LS embedding was well-organized than the existing dictionaries and could lead to improve the F1 score by +1.21%.