# Role Knowledge Prompting for Document-Level Event Argument Extraction
## 4 Experiments
### Main Results and Analysis  
Table 2 shows the performance of our approach and all benchmark models on the RAMS and WIKIEVENTS datasets.  
**Main Results**. Overall, our proposed RKDE framework outperformed all the other methods on the test set. The performance of our proposed RKDE method was the best on both datasets, with the F1 score of Arg-C obtaining 3.2% and 1.4% improvement over the state-of-the-art model BART-Gen (BART-Large) on the RAMS and WIKIEVENTS datasets, respectively. We used BART-Base in the experiments. If compared with BART-Gen (BART-Base), the performance of our method on both data sets then exceeded it even more with the F1 score of Arg-C obtaining 5.4% and 22.1% improvement on the RAMS and WIKIEVENTS datasets, respectively. This shows the effectiveness of our method.  
**Performance Analysis.** Concretely, we modeled the DEAE task as a slot padding task by extracting all arguments simultaneously, activating the PLM's potential with a role knowledge prompt-tuning paradigm to guide the template precisely:  
1. When comparing the approaches for the MRC and QA tasks (where MRC means machine reading comprehension while QA means question and answer), MRC is usually in the form of fill-in-the-blank completion for the model to determine which hidden word is most likely, depending on the context, while the questions of QA are answered one by one in accordance with the set question templates. For DEAE, these approaches both use the BERT model, which is applicable to natural language understanding. DocMRC and BERT-QA had comparable performance on the RAMS dataset, but  
\begin{table}
\begin{tabular}{l c c c c c} \hline \hline \multirow{2}{*}{**Method**} & **RAMS** & \multicolumn{4}{c}{**WIKIEVENTS**} \\  & **Arg-I** & **Arg-C** & **Arg-I** & **Arg-C** & **Head-C** \\ \hline SpanSel [7] & - & 40.7 & - & 38.3 & - \\ DocMRC [12] & - & 44.3 & - & 42.1 & - \\ DocMRC(IDA) [12] & - & 45.7 & - & 43.3 & - \\ BERT-QA [13] & 46.4 & 44.0 & 54.3 & 53.2 & 56.9 \\ BART-Gen (BART-Base) [16] & 50.9 & 44.9 & 47.5 & 41.7 & 44.2 \\ BART-Gen (BART-Large) [16] & 51.2 & 47.1 & 66.8 & 62.4 & 65.4 \\ RKDE (ours) & **55.1** & **50.3** & **69.1** & **63.8** & **66.8** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Main results on RAMS and WIKIEVENTS datasets (F1 score (%)).  
BERT-QA had a substantial improvement over DocMRC on the WIKIEVENTS dataset. Our approach is similar to that of reading comprehension, which is carried out in the form of span extraction and has a nearly 10 point performance improvement over the former. We conjecture that the performance improvement mainly lies in adding templates to the pretrained model.
2. In comparing the approaches of generative tasks, the generative model BART-Gen and our model both used prompt templates and had similar performances, but our performance was a bit better. We found that role knowledge guidance plays an active role in the templates. Unlike the prompt of BART-Gen, we designed two sub-prompts (i.e., role definition and event description) to help the model understand the semantics better and introduce role concept knowledge and role interaction knowledge into the template to guide the template for more precise prompting. The results demonstrate that role knowledge prompting for DEAE not only helps to locate the arguments accurately, but it also can cope with arguments and roles overlapping.