# Using BiLSTM Networks for Context-Aware Deep Sensitivity Labelling on Conversational Data
## 6 Results  
Micro-averaged Precision, Recall and F1 metrics have been used for performance evaluation as they are widely used in similar sequence labelling tasks [52; 53; 54] and perform better on imbalanced datasets [55]. Table 2 summarises the results for the entire set of experiments carried out, separated in three sections, with the corresponding models and their performance. **EXPERIMENT A: _Model design and word embeddings:_** As a baseline, a CRF model with casing and word morphology features was used. Table 2 shows that all of our proposed models outperform the baseline CRF. Of these, predictably, RWE performs the poorest. GLoVe embeddings deliver a substantial improvement, with BERT embeddings giving the best results across all three metrics, most likely due to its contextualised nature. On aggregate, results indicate that adding a CRF layer improves the performance of all models slightly. In summary, although the increase of the F1 metric is very marginal, there is a consistent improvement throughout all variants in experiment A when introducing a CRF layer. We observe that BiLSTM\({}_{\text{BERT}}+\text{CRF}\) is the best performing CRF-enriched model in regards to model architecture and embeddings. **EXPERIMENT B: _Character embeddings:_** Despite our expectation for the contrary, results revealed that character embeddings cause performance deterioration. Reasons for this may be: (a) that the supplementary trainable parameters increased the model's complexity and learning the underlying correlations between the data points became more challenging, and (b) that the CNN and pooling architecture is perhaps by design unsuitable for this problem setting. The remaining experiments were conducted without CNN character embeddings. **EXPERIMENT C: _Auxiliary features experiment:_** For the third part of the experiment we used POS tags, IC and WordShape as auxiliary features for the classification. Overall, it is observed that the combination of two or more auxiliary features offers a slight performance advantage against single feature models. Yet, the performance metrics when using those features are not dissimilar from the initial BiLSTM + CRF model with BERT embeddings, except when using WordShape features exclusively. Thus, based on the Recall and F1 metrics, we identify WordShape as a better suited auxiliary feature that can be used with BiLSTM + CRF model. Accordingly, we chose to incorporate WordShape features for further evaluation experiments. \begin{table}
\begin{tabular}{c c c c c c c c c c c c c c} \hline \hline  & & \multicolumn{3}{c}{**MODEL**} & \multicolumn{3}{c}{**EMBEDDINGS**} & \multicolumn{3}{c}{**AUX. FEATURERES**} & \multicolumn{3}{c}{**RESULTS**} \\ \cline{3-13}  & & **BiLSTM** & **CRF** & **CNN** & **RWE** & **GLoVe** & **BERT** & **POS** & **IC** & **WS** & **Precision** & **Recall** & **F1** \\ \hline \multirow{8}{*}{**Category**} & 1 & \multicolumn{3}{c}{X} & & & & & & & & 0.8068 & 0.5779 & 0.6581 \\  & 2 & X & & & X & & & & & 0.8986 & 0.9315 & 0.9147 \\  & 3 & X & & & & X & & & & 0.9347 & 0.9599 & 0.9471 \\  & 4 & X & & & & X & & X & & 0.9452 & 0.9689 & 0.9569 \\  & 5 & X & X & & X & & & & & 0.9113 & 0.9337 & 0.9224 \\  & 6 & X & X & & X & & & & 0.9451 & 0.9558 & 0.9504 \\  & 7 & X & X & & & & X & & 0.9568 & 0.9693 & 0.9630 \\ \hline \multirow{8}{*}{**Category**} & 8 & X & & X & & & X & & & 0.9568 & 0.9576 & 0.9572 \\  & 9 & X & X & X & & & X & & & 0.9634 & 0.9558 & 0.9596 \\ \hline \multirow{8}{*}{**Category**} & 10 & X & X & & & X & X & & & 0.9548 & 0.9685 & 0.9616 \\  & 11 & X & X & & & X & & X & & 0.9611 & 0.9618 & 0.9614 \\  & 12 & X & X & & & X & & & X & **0.9640** & **0.9706** & **0.9673** \\ \cline{1-1}  & 13 & X & X & & & X & X & & 0.9639 & 0.9625 & 0.9631 \\ \cline{1-1}  & 14 & X & X & & & X & & X & 0.9635 & 0.9629 & 0.9632 \\ \cline{1-1}  & 15 & X & X & & & X & & X & 0.9628 & 0.9615 & 0.9622 \\ \cline{1-1}  & 16 & X & X & & & X & X & & 0.9611 & 0.9704 & 0.9657 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Model variations and performance metrics of the experiments.