# Generating Chinese Event Extraction Method Based on ChatGPT and Prompt Learning
## 3 Model  
### Generative Event Extraction Framework  
Currently, pre-trained language frameworks have been widely applied in natural language processing. Some major pre-trained language frameworks include BERT (Bidirectional Encoder Representations from Transformers) [30], GPT (Generative Pre-Trained Transformer) [31], and BART (Bidirectional and Auto-Regressive Transformers) [5], among others. Among them, BART combines autoregressive and denoising autoencoder mechanisms, enabling the model to learn text generation and comprehension simultaneously. As a result, it has reliable generation capabilities and finds extensive applications in generative tasks. By defining the event extraction task as a sequence-to-structure text generation task, we can extract trigger words and event arguments from the text in an end-to-end manner, simplifying the complex structure prediction task into a neural network model and linearizing all trigger words and event arguments as structural expressions. For this purpose, this paper uses the BART model as the base model for the pre-trained language.  
The BART model outputs linearized structural expressions \(Y=y_{1},\ldots,y_{m}\) for a given input token sequence \(X=x_{1},\ldots,x_{n}\). The principle of its working process follows.  
First, BART converts the token sequence into embedding vectors through an embedding layer:  
\[X_{emb}=\text{Embedding}(x_{1},\ldots,x_{n}) \tag{1}\]  
Next, the input's hidden vector representations \(H=h_{1},\ldots,h_{n}\) are computed using multiple layers of transformer encoders, where \(h_{i}\) represents the context-aware representation at the \(i\)-th position:  
\[H=\text{Encoder}(X_{\text{emb}}) \tag{2}\]  
In the above equations, \(\text{Encoder}(\cdot)\) represents the computation process of an encoder layer, where each encoder layer consists of a self-attention mechanism and feed-forward neural network layer.  
After the input token sequence is encoded, the decoder sequentially utilizes the hidden vectors of the input tokens to predict the output structure one by one. At the \(i\)-th step of generating text, the self-attention decoder predicts the \(i\)-th linearized token \(y_{i}\) and the decoder state \(h_{i}^{d}\) as follows:  
\[y_{i},h_{i}^{d}=\text{Decoder}\Big{(}[H;h_{1}^{d},\ldots,h_{i-1}^{d}],y_{i- 1}\Big{)} \tag{3}\]  
Here, \(\text{Decoder}(\cdot)\) represents the computation process of a decoder layer, where each decoder layer includes a self-attention mechanism, encoder-decoder attention mechanism, and a feed-forward neural network layer.  
The conditional probability of the entire output sequence \(p(y|x)\) is obtained by multiplying the probabilities \(p(y_{i}|y_{<i},x)\) at each step:  
\[p(y|x)=\prod_{i}^{|y|}p(y_{i}|y_{<i},x) \tag{4}\]  
Here, \(y_{<i}=y_{1},\ldots,y_{i-1}\) and \(p(y_{i}|y_{<i},x)\) is the normalized probability distribution obtained by applying softmax(\(\cdot\)) overall target vocabulary.  
On the BART pre-trained language model, this paper proposes a generative event extraction model based on ChatGPT and prompt learning, called ChatGPT-Prompt-Event Extractor (CPEE), which incorporates improvements from both data augmentation and input construction perspectives. The specific enhancements are shown in Figure 1.  
1. Based on known event knowledge, through the interaction with ChatGPT under the rule template, the event extraction annotation corpus is used to pre-train the model through supervised learning, adapting it to downstream tasks (pre-finetuning).
2. Combining prompt learning and construct input templates for generative models, in these templates they explicitly incorporate entity markers and event knowledge to fully leverage the pre-trained language model's prior knowledge.