# Generating Chinese Event Extraction Method Based on ChatGPT and Prompt Learning
## 4 Experimental Analysis
### Experimental Results  
To evaluate the proposed model (CPEE) in this paper, we compared it with several existing models that have achieved good experimental results in the field of event extraction in recent years, which serve as baseline models. Precision (P), recall (R), and F1-measure were adopted as evaluation metrics. The evaluation method comprehensively assessed trigger words and event arguments collected for each model. Specifically, we compared our model with baseline models using sequence labelling and machine reading comprehension-based approaches. The running experimental environment was Linux, the GPU model was Tesla V100, the memory was 32 G, and the deep learning framework was Pytorch1.12.1 [35]. The experimental parameters were Epoch = 20, Batch size = 16, Max length = 1024, dropout = 0.5, and lr = 0.01. Using the data enhancement method proposed in this paper, 4000 pieces of data were generated based on 3000 randomly selected event trigger words. Subsequently, the CPEE model proposed in this paper was run and compared with other models in the public dataset DuEE1.0 with Title2 Event for experiments, and the results are shown in Table 4.  
Through the analysis of the comparative experimental results, it can be observed that the CPEE model outperforms mainstream event extraction models in terms of F1-score on both Chinese datasets. Specifically, on the DuEE1.0 dataset, the CPEE model achieves a 9.2% improvement in F1-score compared with the baseline model ERNIE + CRF and a 1.3% improvement compared to the MRC model. Although its recall rate is lower than that of the MRC model, the CPEE model performs better by achieving higher precision, indicating its ability to extract accurate event information. On the Title2 Event dataset, compared with the other three models, the CPEE model exhibits significant advantages, achieving good results in precision, recall, and F1-score. It shows an 18.7% improvement in the F1-score compared with the sequence labelling model BERT-CRF and a 10% improvement compared with the Seq2 seqMRC model. This indicates that CPEE is better at comprehensively capturing accurate and actual trigger words and event arguments. To further validate the effectiveness of different models, we conducted ablation experiments on the Title2 Event dataset, and the results are shown in Table 5. The introduction of the ChatGPT-annotated data corpus for pre-finetuning improves the F1-score for the extraction task by 2.2%, while the introduction of semantic annotation results in an F1-score improvement of more than 1.7%. The results indicate that pre-finetuning with the ChatGPT-annotated data corpus is more capable of improving the model's adaptability to the event extraction tasks. Semantic annotation, on the other hand, effectively transfers the knowledge of the pre-trained language model by encoding event labels as natural language \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Model** & **P** & **R** & **F1** \\ \hline \multirow{4}{*}{DuEE1.0} & ERNIE + CRF [36] & - & - & 75.9 \\  & Bert4 keras [30] & 85.5 & 73.7 & 79.2 \\  & MRC [16] & 82.6 & 85.0 & 83.8 \\  & CPEE & 87.6 & 82.8 & 85.1 \\ \hline \multirow{4}{*}{Title2 Event} & BERT-CRF [37] & 41.1 & 41.3 & 41.2 \\  & MRC [16] & 44.5 & 44.8 & 44.7 \\ \cline{1-1}  & Seq2 seqMRC [38] & 49.8 & 50.1 & 49.9 \\ \cline{1-1}  & CPEE & 62.3 & 57.8 & 59.9 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Performance comparison of different models on public datasets. vocabulary. These contributions are of great significance in improving the performance of event extraction tasks. In order to further validate the effectiveness of the ChatGPT-annotated corpus, the experimental data were expanded to different degrees, and different models were compared. Figure 4 shows the experimental results of different event extraction models on the ChatGPT-annotated corpus, using the F1 value as the evaluation index of the experiment and the Title2 Event dataset as the public dataset.