# Prompt Language Learner with Trigger Generation for Dialogue Relation Extraction
## 4 Experiments
### Experimental Results  
#### 4.2.1 Full-Shot Setting  
Table 2 demonstrates the performance change of the models to which our methods were applied compared to the baseline model BERT\({}_{s}\). All our models achieved performance improvement in F1 and F1\({}_{c}\) scores of the development set, and those models showed a similar overall tendency except for the model with the generated triggers (_GenTr_) in the development set. The F1 score of the _GenTr_ model in the development set decreased by 0.14%p compared to the baseline model. In particular, the model with MLM and manual initialization methods (\(MLM+\textit{mini}\)) showed the overall highest performance across the development and test set results. Compared to the BERT\({}_{s}\) model, the \(MLM+\textit{mini}\) model showed 4.32%p and 3.57%p improved performance at F1 and F1\({}_{c}\) score, respectively, in the development set and achieved 3.75%p and 3.31%p of performance gains in the test set. Although the _GenTr_ model showed 1.26%p, 0.74%p of performance gains in the development set at F1 and F1\({}_{c}\) scores, it showed only a slight increase or decrease with the amount of approximately 0.1%p in the test set. Moreover, when comparing _GenTr_, \(MLM\) and \(GenTr+MLM\) models, in the development set, the model in which the MLM method and the generated triggers were used together (\(GenTr+MLM\)) showed the best performance with 64.92 of F1 and 59.44 of F1\({}_{c}\) scores, whereas in the test set, the model in which only the masked-language modeling (MLM) method was used (\(MLM\)) performed the highest with 62.27 and 57.12. Figure 2 illustrates the difference between the F1 score and F1\(c\) score for each model. This can be interpreted as the smaller the deviation, the more consistent the performance \begin{table}
\begin{tabular}{l c c c c c c c c c c c c} \hline \hline  & \multicolumn{10}{c}{**Full-Shot Setting**} \\ \hline \multirow{2}{*}{**Method**} & \multicolumn{5}{c}{**Dev**} & \multicolumn{5}{c}{**Test**} \\ \cline{2-13}  & **P** & **R** & **F1** & **P\({}_{c}\)** & **R\({}_{c}\)** & **F1\({}_{c}\)** & **P** & **R** & **F1** & **P\({}_{c}\)** & **R\({}_{c}\)** & **F1\({}_{c}\)** \\ \hline BERT\({}_{s}\)\({}^{*}\) & 61.14 & 62.50 & 61.81 & 63.84 & 51.27 & 56.86 & 59.46 & 60.58 & 59.94 & 63.33 & 49.09 & 55.29 \\ \hline +_GenTr_ & 62.91 & 63.24 & 63.07 & 65.00 & 51.72 & 57.60 & 59.00 & 60.68 & 59.80 & 63.20 & 49.45 & 55.48 \\ +_MLM_ & 62.88 & 64.20 & 63.52 & 65.59 & 52.94 & 58.58 & 62.27 & 62.29 & 62.27 & 65.43 & 50.69 & 57.12 \\ +_GenTr_ +\(MLM\) & 65.09 & 64.76 & 64.92 & 67.07 & 53.38 & 59.44 & 61.13 & 60.95 & 61.03 & 65.42 & 50.13 & 56.76 \\ +_MLM_ +miniit & **65.47** & **66.81** & **66.13** & 67.94 & **54.43** & **60.43** & **64.00** & **63.44** & **63.69** & 67.04 & **52.05** & **58.60** \\ +_GenTr_ +\(MLM\) +miniit & 64.92 & 66.35 & 65.62 & **67.98** & 53.99 & 60.18 & 61.18 & 62.67 & 61.90 & **67.19** & 50.86 & 57.87 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: This table shows the main experimental results in a full-shot setting. The \({}^{*}\) mark indicates the re-implemented version. P and R indicate precision score and recall score, respectively. In addition, F1\(c\) score is the F1 score in the conversational setting. The best performance is bold and the second best is underlined. is in conversation settings. First, in the validation set on the left, the results of all models except the \(MLM\) model are located above that of the baseline (BERTs). In contrast, in the test set on the right, the difference between the F1 score and F1\(c\) score of the \(MLM\) model and the \(MLM+minit\) model is larger than that of the baseline, and the differences of the other models are observed to be located under the baseline. According to these results, the results of the validation set may make it seem as if the impact of the provision of triggers and prompt-based approaches is inconsistent in conversation settings.