# Context-Aware Bidirectional Neural Model for Sindhi Named Entity Recognition
## 5 Results and Analysis
### Final Results  
The baselines and CaBiLSTM models had several components that we could tweak to understand their impact on the overall performance. We analyzed the effect of pretrained SdGloVe, SdfastText representations, task-specific character-level, word-level representations, self-attention, and softmax, CRF classifiers. It can be observed in the performance comparison of baselines and the CaBiLSTM model that F1-score obtained by CaBiLSTM (see Table 7) is higher than all the baselines for three types of representation learning. The word-level and character-level representations almost yield close results to each other on the GloVe. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{**CWS**} & \multicolumn{3}{c}{**No Dropout**} & \multicolumn{3}{c}{**Dropout = 0.25**} & \multicolumn{3}{c}{**Dropout = 0.5**} \\ \cline{2-10}  & **P\%** & **R\%** & **F\%** & **P\%** & **R\%** & **F\%** & **P\%** & **R\%** & **F\%** \\ \hline
3 & 88.74 & 89.41 & 88.43 & 89.67 & 90.28 & 89.55 & 88.24 & 88.39 & 89.17 \\
5 & 90.21 & 90.47 & 90.36 & 90.86 & 90.61 & 90.83 & 89.37 & 89.42 & 89.28 \\
7 & 90.79 & 90.94 & 90.89 & **91.43** & **91.76** & **91.25** & 90.79 & 90.29 & 90.59 \\
10 & 90.84 & 90.9 & 90.82 & **91.13** & **91.32** & **91.28** & 90.36 & 90.72 & 90.24 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: The impact of context (length of CWS) and various dropout rates for the CaBiLSTM model on the SiNER test set using the contextual representation learning approach. The bold results highlight the best performance. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Representation** & **Model** & **Variants** & **Precision\%** & **Recall\%** & **F1-Score\%** \\ \hline \multirow{8}{*}{SdGloVe} & WLSTM & softmax & 84.41 & 84.85 & 84.73 \\  & WBiLSTM & softmax & 86.33 & 85.76 & 85.92 \\  & CBiLSTM & softmax & 86.74 & 85.9 & 86.15 \\  & WBiLSTM & CRF & 87.19 & 86.39 & 86.79 \\  & CBiLSTM & CRF & 87.64 & 86.71 & 86.83 \\  & WBiLSTM & Attention, CRF & 86.59 & 87.46 & 87.32 \\  & CBiLSTM & Attention, CRF & **87.93** & **87.41** & **87.61** \\ \hline \multirow{8}{*}{SdfastText} & WLSTM & softmax & 85.23 & 85.94 & 85.24 \\  & WBiLSTM & softmax & 87.62 & 86.34 & 86.58 \\  & CBiLSTM & softmax & 87.84 & 86.55 & 87.16 \\  & WBiLSTM & CRF & 87.69 & 87.8 & 87.63 \\  & CBiLSTM & CRF & 88.36 & 88.72 & 88.24 \\  & WBiLSTM & Attention, CRF & 88.89 & 88.53 & 88.79 \\  & CBiLSTM & Attention, CRF & **89.45** & **89.42** & **89.27** \\ \hline \multirow{8}{*}{Task-specific} & WLSTM & softmax & 86.48 & 86.63 & 85.82 \\  & WBiLSTM & softmax & 87.61 & 86.28 & 86.9 \\ \cline{1-1}  & CBiLSTM & softmax & 88.35 & 87.59 & 87.26 \\ \cline{1-1}  & WBiLSTM & CRF & 88.62 & 88.91 & 88.58 \\ \cline{1-1}  & CBiLSTM & CRF & **89.34** & **89.38** & **89.21** \\ \cline{1-1}  & WBiLSTM & Attention, CRF & **89.45** & **89.29** & **89.42** \\ \cline{1-1}  & CBiLSTM & Attention, CRF & **90.48** & **90.52** & **90.39** \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Results of neural baseline models on the SiNER test set on the pretrained and task-specific character-level and word-level representations. Bold font denotes the best results. In contrast to SdGloVe, the character-level representations achieve better performance than word-level representations in SdfastText. However, the task-specific character-level and word-level representations achieve the best performance. It is because representation learning with the BiLSTM model captures the contextual features. The presented results also demonstrate that the CRF is dominant over softmax. The CRF gains +0.87% and +0.58% on the word-level WBiLSTM and character-level CBiLSTM models with SdGloVe representations. Similarly, CRF is also beneficial in SdfastText and task-specific representation learning, respectively. Moreover, self-attention has boosted the F1-score in all the experiments. The task-specific word-level, as well as character-level representations, are dominant along with self-attention and CRF.