An Iteratively Parallel Generation Method with the Pre-Filling Strategy for Document-level Event Extraction
Authors: Guanhua Huang
Runxin Xu
ying zeng
Jiaze Chen
Zhouwang Yang
Weinan E
Venue: EMNLP 2023 Main (2023)
Keywords: natural language processing
information extraction
document-level event extraction
parallel generation

Abstract: In document-level event extraction (DEE) tasks, a document typically contains many event records with multiple event roles. Therefore, accurately extracting all event records is a big challenge since the number of event records is not given. Previous works present the entity-based directed acyclic graph (EDAG) generation methods to autoregressively generate event roles, which requires a given generation order. Meanwhile, parallel methods are proposed to generate all event roles simultaneously, but suffer from the inadequate training which manifests zero accuracies on some event roles. In this paper, we propose an Iteratively Parallel Generation method with the Pre-Filling strategy (IPGPF). Event roles in an event record are generated in parallel to avoid order selection, and the event records are iteratively generated to utilize historical results. Experiments on two public datasets show our IPGPF improves $11.7$ F1 than previous parallel models and up to $5.1$ F1 than auto-regressive models under the control variable settings. Moreover, our enhanced IPGPF outperforms other entity-enhanced models and achieves new state-of-the-art performance.

An Iteratively Parallel Generation Method with the Pre-Filling Strategy for Document-level Event Extraction
Guanhua Huang', Runxin Xu², Ying Zeng', Jiaze Chen"!, Zhouwang Yang' and Weinan E²
Uniiersitty of Science and Technology of China Peking University 'Bytedance guanhuahuang@mail. www.usttcco.. edu. en, runxinxu@gmail.. com, zengying. sesbsssssyyeeee teelyyddddccccommm com teoyde@gmail com, yangzweustc. edu. cn, weinanêmath. pku, edu. cn
Abstract
Decument
Event Record
[86] Jingrons Group." incressed is hilddnns the company's 's stock by 182 shares through the secondary market on Dec 15, 2011....
Enily Martion
In document-level event extraction (DEE) tasks, a document typically contains many event records with multiple event roles. Therefore, accurately extracting all event records is big challenge since the number of event records is not given. Previous works present the entity- based directed acyclic graph (EDAG) genera- tion methods to autoregressively generate event roles, which requires given generation order. Mcanwhile, parallel methods are proposed to generate event roles simultaneously, but suf- fer from the inadequate training which mani- fests zero accuracies on some event roles. In this paper, we propose an Iteratively Parallel Generation method with the Pre-Filling strat- egy (IPGPF). Event roles in an event record are generated in parallel avoid order selec- tion, and the event records are iteratively gen- crated to utilize historical results. Experiments two public datasets show our IPGPF im- proves 11 7 Fl than previous parallel models and up to 5.1 Fl than auto-regressive models under the control variable settings. Morcover, our enhanced IPGPF outperforms other entity- enhanced models and achieves new state-of- the-art performance
bipilyyheder
Jingzong Grap
TradidSture
A82 slares
hold
Jiggoong Group jan the Company increased from 90 880,8000 shares 11..620088 shares
Event Typa
Laterihioling 91 91862266 slanes
Endiate
Doe 15, 2011
Shares
AvrrapePriv
N.L.L.
[89] on Dec 16, 2011, Jinggrong Group reduced ins holdings 35,000 shures, with an average price of 19.88.
Event
EquisyHoMMs Janasoe Google
Argunent
TradedShares 35yj4
Startbute
announecment, linggong Group holds 91 1,22 (158 shares of the company.
Role..
Kadiiass
Deo 16, 2011
LakerHoMingg vi 317 J0S8 chone
Skarss
AvergePriice 19.88
Event Record Generation
Equitytoller
TradeShares
AveragePrice
Auto- regressiee
Equitytoller
Tradeshares
AveragePriie
Equityliolder
TradeShaares
AverapePrice
Iteratiully
Parallal
[ours]
Pre Eilling
Figure I: An example of document-level event extrac- tion and different event record generation methods.
1 Introduction
Document-level event extraction (DEE) aims to extract multiple event records from the entire docu- ment (Ebner et al., 2020; Du et al., 2021b; Li et al., 2021; Xu et al., 2022). Different from sentence- level event extraction (SEE) (Chen et al., 2015; Nguyen al., 2016; Du and Cardie, 2020b), event arguments of an event record are usually scattered across multiple sentences, while overlapping argu- ments contained in several event records appear more often. Moreover, real-world event records
always use similar but different words to report the same type of events, lcading to low annota- tions of triggers and the requirement of trigger-free methods. The absence of triggers increases the difficulty of DEE. Therefore, extracting multiple event records without triggers is the main challenge of the trigger- free DEE methods.
Most free DEE methods (Zheng et al., 2019; Xu et al., 2021a: Liang al., 2022) build entity- directed acyelic graph (EDAG) to auto- regressively generate event arguments with their roles under a predefined order. However, it is im- portant to note that determining the optimal order
'Work was done when Guanhua was an intern ByteDance Al Lab.
"Corresponding author. 'Our cnde is available at httpp:://iithu com/ CarIanLark/IPGPF

---

from the permutation space of possible orders has an NP-hard complexity of ( Parallel mod- els (Yang et al., 2021) are presented to generate all event records and roles simultaneously to avoid the error broadcasts in a given event role order. Nev- ertheless, parallel models are particularly prone to the inadequale training, leading to zero accuracies certain event roles.
entity in the document which refers a named entity; (3) event argument: an entity playing a
specific event role in event extraction; (4) event role: a predefined argument type corresponding to event arguments; (5) event record: the expression of an event that contains many event arguments with their event roles.
A trigger-free doc-level event extraction task usually includes several pipeline sub-tasks. Fol- lowing (Zheng et al., 2019), IPGPF finishes the event extraction by handling three sub-tasks: (1) Named Entity Recognition (NER): extracting en- tity mentions as argument candidates from the doc- ument: (2) Event Detection (ED): judging whether there exist predefined event types in the document. (3) Event Record Generation (ERG): generating event records type-by-type. Besides, the model must be able to generate multiple records for one specific event type since there is no trigger.
In this paper, we propose an Iteratively Parallel Generation method with the Pre-Filling strategy (IPGPF). As shown in Figure 1, event roles in an event record are generated in parallel, thereby avoiding the role order selection. Meanwhile, event records are iteratively generated to leverage the his- torically generated records. The pre-filling strategy is further designed for the inadequate training in parallel event role generation. Specially, the pre- filling strategy adaptively pre-fills several event roles by the results in historically generated event records before the parallel event role generation, and then the rest roles are generated with the help of fre-filled roles.
Due to the space limit, we put the details of NER and ED sub-tasks that are the same with (Zheng et al., 2019) Appendix A.3 and A.4.
Moreover, we explore the effect of event role orders on auto-regressive EDAG models. Exper- iments show that a change in event role orders in these models can bring 3.7 Fl decrease over a whole dataset and 10.4 Fl decrease in multiple events scenarios.
Methodology
Given a document D containing N, sentences { doc-level event extraction aims to gener- ate multiple event records Z - } where N₂ is the number of ground truth event records of the document. An event record consists of n event guments 2 (at af...., ..., a²) and their event roles (r). r ........)) As shown in Figure 2, IPGPF first extracts entity mentions as candidate arguments through named entity recognition (Appendix A.3). The presence of predefined event types is then judged by event detection (Appendix A.4). Finally, event roles in event record are generated in parallel, while event records are generated iteratively (Section 3.1). To alleviate the inadequate training in parallel gener- ation, we propose an effective pre-filling strategy (Section 3.2).
We summarize our contributions as follows: (1) We propose IPGPF, an iterative generation method that eliminates the need for selecting the order of event role generation. achieves this through two-stage matching that refines generated event records by leveraging historical results. (2) We design the pre-filling strategy, which success- fully alleviates the inadequate training and zero- accuracy problems in parallel event role generation. (3) Experiments on two large-scale datasets show IPGPF gets 11.7 Fl performance gain than paral- lel methods and outperforms 1.4 to 5.1 Fl than auto-regressive generation methods control vari- able comparison. Additionally, IPGPF is flexible enough to incorporate entity enhancement and out- performs other enhanced models.
A symbol reminder is listed in Table 7 for better understanding of our method.
3.1 Iteratively Parallel Generation
2 Preliminaries
After the NER task and ED task, obtain the candidates arguments representation Hº R(N++1)xd and document sentence representation Ha E NR x. where N, is the number of candidate arguments, N₂ is the sentence number, and is the hidden dimension. Then we generate event records
We first clarify several important concepts: (1) named entity: mentions of rigid designators from text belonging to predefined semantic types such as person, location, organization etc (Nadeau and Sekine, 2007); (2) entity mention: a text span of

---

Event Detection
Sentences Emberdting
Semnnnce festire
EqunyFroeze: E qqunyünderweigtt
Embelding
Feature
Becord-1
Grand Tnith Fvent fiecodss
Tobenn
MXX
parsilal template filing
gready role matching
fitered mocrd mutching
Jingzong Group sells 35,0000
Add
Document
sltares... price of 19,88
Decoderi
Becord-2
2
Record-2
Record-1
EquityHoller sells Tradedohares AvorggePrice
y
Predillngg
y
iberative generation
event record hiltering
Named Entity Recognition
Event Record
The overall architecture of IPGPE. Given document, IPGPF first extracts entity mentions candidate arguments. Then the existence of event types is detected. Finally, event roles are generated from candidate arguments by filling queries manual template in parallel, and event records are iteratively generated. two-stage many-to-one matching loss is further designed for the training of IPGPE
Figure 2:
iteratively and generate event roles a record in parallel.
3.1.2 Iterative Generation
To better use the generated historical event records, we design an iterative generation method.
3.1.1 Parallel Template Filling
As shown Figure 2, in the iteration, historical event record is first represented by combining the template queries and extracted arguments:h" = M a.r Pooling( H" Q'I) IRd
Parallel event role generation is an excellent way to avoid unstable performance due to role order selection.
To further help event role generation, we manu- ally build templates for each event type. As shown in Figure 2 and Figure 8, event roles are represented by special tokens in these templates. So we can generate event records by filling the special event role tokens in given templates.
At (i + 1)-th iteration, all historical records a [h] he, ...,..]] hi] с Rixd are concatenated the template queries in Equation (1) for the tem- plate filling task of next iteration:
[H* H* Decoder (IQ', H²) Hº)
(3)
In event record generation of a special event type, we first embed the corresponding template as Q' igi+ 26.x-.xxx E R.N where N, is the number of template tokens. Then a vanilla transformer decoder is employed to encode the template query:
3.1.3 Event Record Filtering
As shown in Figure 2, to extract all event records in document through iterative generation, the it- eration number is usually larger than the number of ground truth event records. Therefore, filter is required to filter several output records final results.
H Decoderr(Q" .11)
(1)
where H' E Nexd Ha E +1)xd is the hidden representation of candidate arguments in Equation (17). Next, we leverage a pointer net- work (Vinyals et al., 2015):
After N iteration, we get event record represen- tation H = [h hi R c Nix, Then we filter several best records as the final outputs by utilizing a vanilla transformer decoder and a linear layer classifier:
pr = Softmar(tanh (H"W" H²W"). (2)
A = 2ecoder2(HH. H*)
(4)
where H" E xx is event role representation from Hº W", Wo E Rdxd E Rd are trainable parameters, + is the broadcasting plus of two ma- trices, pr C R (NN is the score of arguments corresponding to roles in the event type.
p Sigmoid .W*
(5)
where W E dd are trainable parameters, pa IR. N: are scores output event scores.
3.1.4 Two stage Matching
Finally, we extract arguments Ha E IR X. cor- responding to event roles.
Different from (Yang et al., 2021) which use a one- stage one-to-one matching to train their parallel

---

model, we design a two-stage many-to-one match- ing algorithm to first performs multiple iterative refinements on the generated event records, and subsequently filters the best results.
L.Choose
Sugpang
record
Corma Aegament
record-1
Tampsinss Contact
Ueseeree Aepunent
recerdi2
Jinggonn Group
Sells
35 ,000
shares
NULL
In the model training, following (Zheng et al., 2019), we generate event records for each event type independently and finally sum the loss of all event types. In this subsection, we will describe the training loss definition for one type in detail.
record.t
0.95
0.75
0.75
Ix 0-0.7
+
recordd11
Jingzong Group
TraderShores
AverogePoce
rewordd22
Greedy Role Matching Given extracted argu- ments score Y = [PT, Ps,..., 12] from Equa- tion (2), and the ground truth arguments y N ".,,Y,,,,,,
Decoder1
O
retzord-n
Jingzong Gromp
Sels
record: [t+3]
[VT, .2,-..,y and vj is the golden label refer to the j-th role in i-th event record. We define a cost function and pairwise compute the cost between output record roles Y/ P. R.N,x(N, +1) and ground truth record YJ R.V:;
where y = (ví (w,19,,.... -b,....,bhh V'x,
0.98
0.97
0.89
not role loss
compute role loss
Figure 3: An example of the pre-filling strategy. At +++ 1)-th iteration step, a historically generated record first chosen. Then some event arguments are pre- filled to corresponding event role queries in template, Finally, the pre-filled template is fed to a Decoder to fill the rest event role queries.
Crate (2.'
Σ Mr log P.,
(6)
We build greedy many-to-one matching to iter- atively refine the event records multiple times, by assigning the most similar ground truth record for each Y as its label:
We find the minimal cost matching x² by the hungarian algorithm (Kuhn, 1955) (Appendix A.5). Then assign label for records M that matched the ground truth record and label 0 for unmatched records, and get the binary cross-entropy loss:
j arg min Crate (2,", .Y)
Levent
log +
Σ log (1 - p() (11)
Then we compute cross-entropy loss for event role generation:
ieM
REM
Finally, we combine the role loss and event loss:
N No ΣΣ
Lrole
Y ( (i),kk
log P.
(8)
LERG = 7 Crote + 2eeveent
(12)
k=1
where T is a surjective function, which computed as mentioned in Appendix A.5
where 71, 72 (0, hyperparameters.
3.2 Pre-filling Strategy
Filtered Record Matching Given the filter scores / p (pf på. .T,.),) R:N: of output event records, and note that the corre- sponding label from ground truth record y
Given document D, the goal of DEE model is fitting the joint distribution P(y1 y2. ... ynd D) of all event roles {yr However, previous parallel methods attempt to learn this joint distribution di- rectly, resulting complex and high-dimensional loss landscape, which can susceptiblely lead to in- adequate training with failure optimization.
(vi Vú. ..., .f......... vi. =II Noc, we pairwise compute the cost value between i-th output record Y = pt and j-th ground truth record Y = uf:
Cevent (Y, = y log p3 log p
To address this issue, we propose a pre- filling strategy to convert the joint distribution P(y, 92, ..., yn yn to conditional distribution P 4p)l x P(y,, & Lpr] y1, Ipf). where Tpf represents the index of pre-filled event roles. By doing so, the loss landscape of the condi- tional distribution is dimensilmmnsional, bringing a
To filter several best output records as the final results and abandon the rest, we combine the event filter cost and role cost:
Cau (Y,, Y2) = Cevent (vi",))) + Crote (Ys",, .Y) (10)

---

better parameter optimization. We provide an ex-
are extracted by a pre-trained encoder and used for the next ERG. Finally, we get our simple entity- enhanced IPGPF model (IPGPFFt) which has better model performance.
perimental optimization analysis in Scction 4.6.3.
As shown Figure 3, at (t + 1)-th iteration, we first check the filtered output event records p
.pi.Pá....... PÊ) = IR' computed by Decoder2 in Equation (4), and categorically sample one histori-
4 Experiments
cal record Y: by probability:
4.1
Dataset and metrics
pp Tip > (1) (1-
The ChFinAnn dataset (Zheng et al., 2019) is large dataset focuses on five event types from the financial text, which has 25, 632/3, 204/3, 204 for the train/dev/test set. According to statistics, a document in ChFinAnn consists of 20 sentences and has 912 tokens on average, and 29% of 32, 040 documents in ChFinAnn contain multiple event records.
(13)
1pp > a
where a is the binary classification threshold.
After choosing an output event record, at (t 1)- th iteration, we use a Bernoulli sampler to se- lect each correct predicted argument whose score p² > 3 with probability k, where 3, K E (0, 1) are hyperparameters. Then we pre-fill these arguments to corresponding event role queries in the template. Finally, the pre-filled template is fed to Decoderl fill the rest event role queries, and only the loss of not pre-filled queries are computed.
DuEE-fin (Han et al., 2022) contains 13 event types, which has 7, 015/1, 171/59, 394 documents for the train/dev/test set. Especially, the inference results of the test set need to be submitted online for its evaluation. Only 3, 513 documents in the test set are actually evaluated, while the rest 55, 881 documents are given to prevent manual prediction. Following (Zheng et al., 2019), we use the miero precision, recall, and Fi-score over all arguments.
It is notable that our pre-filling strategy is uti- lized to make a better optimization for model train- ing, inference does not need pre-filling since the model is already trained. Thus, we use ground truth records to select the correct arguments for pre-filling in the model training and do not need them for inference.
4.2 Models for Comparison
Doc2EDAG (Zheng et al., 2019) designs an auto-regressive entity-based directed acyclic graph (EDAG) to generate event records. Before the gen- eration, Doc2EDAG uses two transformer encoders to get features of arguments and sentences. DE- PPN (Yang al., 2021) uses the same encoders with Doc2EDAG to obtain the features of argu- ments and sentences, but generates event roles in parallel. SCDEE (Huang and Jia, 2021) builds an enhanced entity-sentence community graph, then detect event records from the graph and extract entities arguments corresponding to the event roles.PTPCG (Zhu et al., 2022) builds an entity- based pruned complete argument graph with addi- tional entities. It first selects pseudo triggers and then make a beam extraction based on the graph. GIT (Xu et al., 2021a) and RAAT (Liang et al., 2022) follow the same auto-regressive generation with Dec2EDAG, but add a heterogeneous graph network and entity relation extraction network to enhance the entity representation, respectively.
3.3 Learning
For training our IPGPF model, we make a weighted summing over losses of three sub-tasks:
CALL = AIENER + ARLED + AzGang (14)
where LNER, LERG are losses of NER, ED and ERG, respectively, and λ1, A2, λa are hyperpa- rameters of corresponding losses.
3.4 Entity Enhancement
The first sub-task of the trigger-free DEE task is NER, which enables ERG generate events by se- lecting arguments from extracted entities. However, in the training data, entity labels are the same as the event argument labels, which brings some entity la- bel conflict issues. For example, the NER module can easily recognize "Dec 15, 2011 1" as a date but confuses about whether it belongs to "StartDate" "EndDate", leading the error propagation. Thus, we merge several entity labels with the same meaning to simplified type for the NER task. The details of our merge rules can be found in Appendix A.1 Then these simplified entities
To investigate the impact of different role orders in auto-regressive models on performance, we con- duct a comparison between models with human- selected orders

---

HS, using orders in their papers) and models with random-sclected orders (Dos2EDAG-RS/GIT RS/RAAT-RS, reporting the average results of random orders). The detailed orders can be found in Appendix A.13.
Overall results Table presents an overview of the performance of our method on two large-scale public datasets, namely ChFinAnn and DuEE-fin. In the control variable comparison, our IPGPF significantly outperforms the parallel model DE- PPN by 3.2 Fl on ChFinAnn and 11.7 Fl on DuEE- fin, which is analyzed in detail in Section 4.6. This indicates that our iteratively parallel gener- ation with the pre-filling strategy is better than the pure parallel method. Compared with the auto- regressive model Doc2EDAG, our parallel model IPGPF achieves a 1.4 FI improvement over the human-sclected order and 3.8 F1 improvement over the average results of randomly selected or- ders on ChlinAnn. This illustrates that our iterative parallel generation method more robust than pre- vious auto-regressive methods.
Given that ERG is the most important and chal- lenging subtask in doc-EE task, we conduct control variable comparison of Doc2EDAG, DE- PPN and our proposed IPGPF. These models use the same NER and ED modules but different ERG module. We aim to fairly compare our pro- posed iteratively parallel method with previous auto-regressive and parallel methods.
It is worth noting that although our iterative par- allel generation method is mainly proposed for ERG, it is flexible combine with other NER and ED modules. We also compare our enhanced IPGPF with other entity-enhanced models, includ- ing SCDEE, PTPCG, GIT, and RAAT, by incorpo- rating a simple entity enhancement mentioned in Section 3.4.
After adding a simple entity enhancement, our IPGP outperforms other entity-enhanced models on both ChFinAnn and DuEE-fin test sets. When compared to the previous state-of-the-art model RAAT, our IPGPF outperforms RAAT-HS, which uses human-selected orders, by 0.2 slight better Fl on ChFinAnn and 2.0 Fl on DuEE-fin. Moreover, our IPGPF achieves significant 1.6 improvement on ChFinAnn and 2.2 improvement on DuEE-fin over RAAT-RS, which reports the average performance of randomly selected orders. This reveals that our iteratively parallel generation method is flexible in combining additional features to achieve better performance than stronger SOTA models.
4.3 Experiment Settings
We replicated the compared models by their official released code, except for SCDEE which not open source. Instead, we used the results reported in the SCDEE paper. We provide the details of settings and replications in Appendix A.10.
4.4 Main Results
ChlinnAn
DulLi-fi R
Models
It is worth noting that although RAAT-HS im- proves by 1.4 on ChlinAnn compared to RAAT- RS, it only improves by 0.2 on DuFf--in, demon- strating that artificially finding significantly better order than randomly selected orders is not trivial.
R
F
P
Control Variable Setting
Doc2EDAG-HS
79.0
73.8
76.3
Dac2EDAG-RS
81.4
67.8
73.9
67.9
46.8
55.5
DE-PPN
76.8
72.3
74.5
56.6
38.4
45.8
IPGPF (ours)
82.0
73.8
77.7
64.8
51.7
57.5
Entity Enhanced Setting
Single v.s. Multi records To analyze the model's performance on single and multiple event scenarios, divided the ChFinAnn test set into two subsets. A document in the single-record contains only one event record, while document in the multi- record set contains multiple records.
SCDEE
87.2
72.0
78.9
PTPCG
83.4
74.1
78.5
65.7
54.1
59.3
GIT-HS
82.8
76.5
79.6
GIT-RS
83.8
72.7
77.9
65.4
53.0
58.7
RAAT-HS
82,9
79.3
81.1
69.2
57.4
62.8
RAAT-RS
84.5
75.3
79.7
67.4
58.6
62.6
IPGPF (ours)
85.7
77.3
81.3
68.2
61.8
64.8
As shown in the Table 2, IPGPF/IPGPF* outper- forms other models in most single-record scenar- ios. Although IPGPF/IGPF scored lower than DDc-EDAAGHHSSGGTT-HAAAAATE
Table scores on the ChFinAnn test set and the DuEE-fin online test set. Doc2EDAG-HS/CE HS/RAAT-HS means the results of human selected or- ders in their paper. means the average scores of 5 randomly selected orders for auto-regressive models. Our IPGPF get better per- formance both on the control variable comparison and the enhanced comparison.
on multiple- record scenarios, these auto-regressive models are highly sensitive the generation order. For in- stance, Doc2EDAG-RS reported decrease of 6.8 in multi-record Fl on average compared to Doc2EDAG-HH after randomly changing the role

---

Models
EF
ER
BU
EO
EP
Overall
S.
M.
S.
M.
S.
M.
S.
M.
S.
M.
S.
M.
Control Variable Setting
Doc2EDAG-HS
74.8
60.7
90.5
68.5
70.1
55.9
74.4
63.2
83.1
69.1
84.2
67.4
Doc2EDAG-RS
75.4
58.2
88.0
65.0
74.5
58.8
75.5
64.7
85.6
60.1
84.7
60.6
DE-PPN
70.5
58.5
90.8
66.4
70.8
53.7
68.4
53.4
82.4
66.0
83.3
64.0
IPGPF (ours)
82.0
57.5
93.9
68.5
76.1
60.5
79.9
58.2
85.2
67.6
87.6
66.0
Entity Enhanced Setting
SCDEE
88.7
65.8
PTPCG
81.4
69.0
93.1
63.6
79.5
72.7
83.1
64.5
87.5
69.2
89.2
68.1
GIT-HS
82.8
65.9
91.1
70.2
79.8
66.4
79.4
69.0
85.8
72.4
87.0
71.3
GIT-RS
83.7
63.1
91.7
66.9
80.6
63.4
80.1
65.9
86.7
60.3
88.1
RAAT-HS
75.4
66.6
93.4
73.0
80.0
68.5
78.2
74.9
86.6
74.4
87.7
73.5
RAAT-RS
79.6
65.0
93.4
72.5
79.6
69.3
79.9
71.4
69.1
IPGPF (ours)
85.2
64.7
96.8
67.7
82.4
68.6
83.5
66.0
88.3
71.1
91.0
70.1
Table 2: Comparison of performance on single-record set (S.) and multi-record set (M.) on ChFinAna. *-HS means the results of autoregressive models with human selected orders in their paper, while *-RS means the average scores of 5 randomly selected orders. Auto-regressive models (*-RS) gets crash decrease on the multi-record set after changing the generation order of (*-HS). IPGPF significantly outperforms other parallel models, and get more robust performance than auto-regressive models (*-RS). S.: single-record set; M.: multi-record set.
order. Compared with Dec2EDAG-RS/ / GIT-RS/ / RAAT-RS which report the average results of the generation orders, our IPGPF/IPPF** get signifi- cant improvement on both single record scenarios and multiple record scenarios. These comparison results demonstrate the effectiveness and robust- ness of IPGPF in both single-record and multiple- record scenarios.
the removal of the pre-filling strategy results in a higher-dimensional loss landscape, which can lead to optimization failure and zero accuracies many event roles. This ultimately leads to lower overall performance. We provide detailed analysis of this experiment Section 4.6.
We also conduct the ablation study of the Effect of Templates (Appendix A.6) and the Effect of Iteration Number (Appendix A.7).
4.5
Ablation Study
4.6 Analysis
Model
P
R
S.
M.
In this section, we want to answer the most im- pormant questions in the comparison of auto- regressive, parallel and iteratively parallel meth- ods: (1) What is the effeet of different orders for auto-regressive models? (Sec 4.6.1) (2) What is the inadequate training problem of paralle methods? (Sec 4.6.2) (3) How does our proposed method ad- dress the inadequate training problem? (Sec 4.6.3)
IPGPF
82.0
73.8
77.7
87.6
66.0
matching
-1.2
-3.2
-2.3
-1.1
-3.3
pre-filling
-23
-14.3
-9.6
-11.4
-6.9
IPGPF
85.7
77.3
81.3
91.0
70.1
-matching
-1.7
-42
-3.1
-1.9
-5.1
-pre-filling
+5.8
-47.0
-35.8
-28.8
-45.9
Table 3: Ablation study the two stage many-to-on matching and the pre-filling strategy. S.: single-record set; M.: multi-record set.
To better compare the difference between these generation methods, we focus on the control vari- able comparison between Doc2EDAG, DE-PPN and our IPGPF which use the same NER and ED modules but different generation methods.
To reveal the effectiveness of our proposed two stage many-to-one matching, we replace with the one-to-one matching. Then both IPGPF and IPGPF exhibit a significant decrease in perfor- mance. This is because the one-to-one matching approach does not allow for iterative refinement of historical generated event records.
Moreover, we further analyze the speed compar- ison in Appendix A.8.
4.6.1 Effect of Orders
In addition, we evaluate the impact of our pre- filling strategy by dropping it. After removing this strategy, both IPGPF and IPGPF+ receive the per- formance decrease. As discussed in Section 3.2,
To reveal the effect of different event role orders to EDAG models, we list the comparison between Doc2EDAG models with different event role gen- eration orders Tabel 4. After the change of event

---

Model
R
F1
S.
M.
Event Role
IPPF
IPGPE-w/o pre-filing
Do<EDAG-HS
79.0
73.8
76.3
84.2
67.4
TonuPPledgeRRatio
0.0
77.7
0.0 (-17.0)
Pledgee
0.0
14.8
0.0 (-11.)
Do<2EAGG-RS-1
82.5
65.2
73.0
85.5
57.0
Pledger
44.1
51.8
0.0 (-518)
Du22DAG-RS-22
81.5
67.2
73.6
84.3
60.7
EventDate
0.0
67.8
0.9 (-67.8)
Do<2EAGG-RS-3
83.4
66.8
74.2
85.7
59.4
TotalPledgedSiares
56.5
70.1
60.4 (-8.7)
Do22DAG-RS-4
78.0
74.1
76.0
84.3
66.8
CompanyyName
70.0
78.8
71.8
Do<2DAG-RS--5
81.5
65.5
72.6
83.5
58.9
Pledge
67.7
73.4
65.7 [-1.7)
SeljPledgedRatio
67.0
74.5
0.0 (-74G)
*-RS Range
5,4
3.7
2.2
10.4
DisclosureDate
66.9
74.9
0.0 (-74.)
*-RS Average
81.4
67.8
73.9
84.7
60.6
Overall
54.4
70.0
37.5 (-325)
Table 4: Performance of Doc2EDAG models with differ- ent event role generation orders. The highest scores are colored in bold, while the lowest scores are underlined. The performance of Doc2EDAG fluctuates widely under different orders.
Table 5: Event role Fl scores in the event type Equi- ryPledge (EP) of DuEE-fin. IPGPF leverages the pre- filling strategy to eliminate all zero accuracy problems compared with DE-PPN and IPGPF-w/o pre-filling Fl improvement of IPGPF over IPGPF-w/o pre-filling is listed in brackets.
role order, Doc2EDAG-RS4 reduces 3.7 7 Fl than Doc2EDAG-HI on the entire ChFinAnn test set, while Doc2EDAG-RS-1 gets a 10.4 F1 crash than Doc2EDAG-HS on multi-record set. On average, Doc2EDAG models with five random event role or- ders (Doc2EDAG-RS- to Dec2EDAG-RS-S) get 1.9 overall F1 lower than Doc2EDAG-HS and 5.7 Fl decrease on multi-record set. These compar- isons reveal the EDAG models' heavy reliance on event role orders and significant unstable perfor- mance between different event role orders.
Dac20Tan
POFF
IPFFF
Zato-Accurssyy Roles
-35
ChFivinn
100
DuFE-fin
-33
75-
22-
20
25
Epoch
Figure 4: The number of event roles with Fl scores on ChFinAnn and DuEE-fin after the model training.
ing in more zero-accuracy event roles. However, the pre-filling strategy implemented in our IPGPF model solves most of the zero-accuracy problems. The remaining 4 zero-accuracy roles have few data in DuEE-fin, such the "Underweighh/Hooldin Ra- tio" role with only 20 and non-NULL arguments in the training set and dev set, respectively.
4.6.2 Inadequate Training
Although parallel generation methods avoid the fluctuation caused by event role orders, these meth- ods encounter a serious inadequate training which manifests zero accuracies on many event roles.
Table 5 shows the F1 scores of each event role in the EquityPledge (EP) event type on DuEE-fin. For parallel methods, the Fl scores on several event roles of DE-PPN and IPGPF-w/o pre-filling clearly reveal the zero accuracy problems of parallel meth- ods due to the inadequate training which caused by the failure parameter optimization. Fortunately, the pre-filling strategy successfully alleviates the inadequate training and resolves the zero accuracy problems. IPGPF gets good performance on all these aero-accuracy event roles since adding the pre-filling strategy.
4.6.3 Optimization Analysis
To illustrate how our pre- pre-filling strategy alleviate the inadequate training of parallel models in pa- rameter optimization, define the gradient norm ratio Rarrad mazr(lg) | 2 to analyze the gradient for model parameters. In detail, is the gradient of the event role ys's query in these parallel models, such as a in Q' in Equation 1.
11
As shown in the Figure (a)-(c), the event role "TotalPledgeeRattoo" gets zero accuracy for DE-PPN and IPGPF-w/o pre-filling. Its gradient norm ratio decreases around zero at the beginning of model training and fails to escape, while the gradient norm ratio corresponding to "Pledged- Shares" remained consistently larger than zero. Af- ter adding our pre-filling strategy, "TotalPledge- dRatio "'s gradient norm ratio avoids the failure op- timization and keeps larger than zero significantly. leading to a high FI score. More details can be
As shown in Figure 4, even after training, both DE-PPN and IPGPF-w/o pre-filling models have many event roles with zero accuracies. On the Ch- FinAnn dataset, DE-PPN has 4 event roles with zero accuracy out of 35 total roles. On the DuEE- fin dataset, 35 out of 92 total roles have zero accura- cies. Since DuEF-fin has more event roles than Ch- FinAnn, its joint distribution is higher dimensional, making optimization failure more likely and result-

---

found in Appendix A.14.
are also applied. But the requirement of the anno- tations of event triggers are quite hard meet in practice, due to the labeling cost and difficulty. Consequently, trigger- free methods are required for document-level event extraction. For extract- ing multiple events without triggers, a trigger-fre free method is first designed to detect an event in cen- ter sentence and extract the rest of event arguments from surrounding sentences (Yang et al., 2018 8). Then a widely used entity-based directed acyclic graph (EDAG) generation method is proposed better deal with multiple events extraction (Zheng et al., 2019). Several variant methods based on EDAG generation are presented by utilizing more meticulous feature engineering, such as hetero- geneous graph feature (Xu et al., 2021a; Huang et al., 2021) and entity relation feature (Liang et al., 2022). Additionally, a parallel method is proposed avoid the error broadcast in EDAG gen- eration (Yang et al., 2021), and an efficient model is designed to lighten the model and accelerates the decoding speed (Zhu et al., 2022).
We further compare the training loss between our IPGPF and IPGPF-w/o pre-filling on DuFF-fin training set. As shown in the Figure (d), after utilizing the pre-filling strategy, the training loss of IPGPF becomes significantly lower than the loss of IPGPF -w/o pre-filling, which demonstrates that our pre-filling strategy brings better optimization parallel models.
Jal Mode: DE FPN
(C) Model: PEFF wNO prefflling
0.8-
100
The FiN Stape
Taining Freas
Oct IRGAF
650 Loss Comporison
puss wis predt ino
1000 1390
2500
2000
Figure 5: Gradient norm comparison and Training loss
ablation comparison.
4.7 Case Study
6 Conclusion
Please refer to Appendix A.9 for the case study.
Towards the event records extraction in the DEE task, we propose novel Iteratively Parallel Generation method with the Pre-Filling strategy (IPGPF). IPGPF generates event roles in parallel and iteratively generates event records with the help of the pre-filling strategy. Experiments demonstrate that IPGPF successfully alleviates the inadequate training of parallel generation and avoids the event role order selection in auto-regressive generation methods. In the future, we will work on one-stage generation models to avoid the error propagation and time cost of pipeline frameworks.
5
Related Work
Most previous works concentrate on extracting events out of single sentence. Different neural architectures are utilized to extract events, e.g., con- volution network (Chen et al., 2015), recurrent net- work (Nguyen al., 2016), and Transformer-based network (Yang al., 2019). Recently, some works also tried to cast the event extraction task as ques- tion answering task (Du and Cardie, 2020b; Zhou et al., 2021), or sequence-to-sequence task (Xi- angyu et al., 2021). However, these methods can only extract events at the sentence level, which greatly limits their application scenarios.
Acknowledgements
The work is supported by Anhui Center for Ap- plied Mathematics, NSFC Major Research Plan Interpretable and General Purpose Next-generation Artificial Intelligence (No. 92270205), and Ma- jor Project of Science & Technology of Anhui Province (Nos. 202103a07020011, 202203a05020050,,
Therefore, document-level event extraction has been proposed to fully extract events from the en- tire document. Most document-level EE meth- ods are built upon the event triggers, with which they conduct sequence labelling (Du and Cardie, 2020a; Veyseh et al., 2021) or span-based predic- tion (Ebner et al., 2020; Zhang et al., 2020; Xu al., 2022) to identify event arguments in the doc- ument. Sequence-to-sequeen
Limitations
(Du et al., 202 b,b; Li et al., 2021; Huang et al., 2021; Huang and Peng, 2021; Hsu et al., 2022), and question answer- ing (Wei et al., 2021; Ma et al., 2022) paradigms
Based on thousands of words in document, hidden features obtained from named entity recognition, event detection, and event record generation con- sumes a large amount of memories. Meanwhile,

---

the pipeline framework brings more time complex- ity. Consequently, it costs about 30 to 150 hours on 8 x 32GB GPUs to train Doc2EDAG, DE-PPN, GIT, RAAT, or our IPGPF. Thus, the training cost is the main limitation of our work.
data-efficient generation- based event extraction model. In Proceed- ings of the 2022 Conference of the North American Chapter of the Association for Computational Lin- guistics (NAACL).
Nanyun Peng. 2022. DEGREE:
Kung-Hsiang Huang and Nanyun Peng. 2021. Document-level event extraction with efficient end-to-end learning of cross-event dependencies. In Proceedings of the Third Workshop on Narrative Understanding.
References
Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015. Scheduled sampling for sequence pre- diction with recurrent neural networks. In Advances in Neural Information Processing Systems (NeurIPS), volume 28. Curran Associates, Inc.
Kung-Hsiang Hsiang Huang, Sam Tang, and Nanyun Peng. 2021. Document-level entity-based -based extraction as tem- plate generation. In Proceedings of the 2021 Con- ference on Empirical Methods in Natural Language Processing (EMNLP).
Yubo Chen, Liheng Xu, Kang Liu, Daojian Zeng, and Jun Zhao. 2015. Event extraction via dynamic multi- pooling convolutional neural networks. In Proceed- ings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL).
Yusheng Huang and Weijia Jia. 2021. Exploring sen- tence community for document-level event extraction. In Findings of the Association for Computational Lin- guistics: EMNLP 2021, pages 340-351, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Yiming Cui, Wanxiang Che, Shijin Wang. and Ting Liu. 2022. Lert: A anguusticalllymotivate pre-trained language model.
Xinya Du and Claire Cardie. 2020a. Document- levell event role filler extraction using multi- grannlarrtrr contextualized encoding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL).
Harold W Kuhn. 1955. The hungarian method for the assignment problem. Naval research logistics quar- terly.
John Lafferty, Andrew McCallum, and Fernando CN Pereira. 2001. Conditional random fields: Proha- bilistic models for segmenting and labeling sequence data.
Xinya Du and Claire Cardie. 2020b. Event extraction by answering (almost) natural questions. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).
Sha Li, Heng Ji, and Jiawei Han. 2021. Document-level event argument extraction by conditional generation. In Proceedings of the 2021 Conference of the North American Chopter of the Association for Computa- tional Linguistics (NAACL).
Xinya Du, Alexander Rush, and Claire Cardie. 2021a. GRIT: Generative role-filler transformers for document-level event entity extraction. In Pro- ceedings of the 16th Conference of the European Chapter of the Association for Computational Lin- guistics (EACL).
Yuan Liang, Zhuoxuan Jiang, Di Yin, and Bo Ren. 2022. RAAT: Relation-augmenntdd attention transformer for relation modeling in document-level event extraction. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa- tional Linguistics (NAACL).
Xinya Du Alexander Rush, and Claire Cardie. 2021b. Template filling with generative transformers. In Pro- ceedings of the 2021 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics (NAACL).
Yubo Ma, Zehao Wang, Yixin Cao. Mukai Li, Meiqi Chen, Kun Wang, and Jing Shao. 2022. Prompt for extraction? PAIE: Prompting argument interaction for event argument extraction. In Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (ACL).
Seth Ebner. Patrick Xia, Ryan Culkin, Kyle Rawlins, and Benjamin Van Durme. 2020. Multi-sentence argument linking. In Proceedings of the 58th An- nual Meeting of the Association for Computational Linguistics (ACI..
Cuiyun Han, Jinchuan Zhang, Xinyu Li, Guojin Xu, Weihua Peng, and Zengfeng Zeng. 2022. Duee-fin fin: A large-scale dataset for document-level event extrac- tion. In CCF International Conference on Natural Language Processing and Chinese Computing, pages 172-183. Springer.
David Nadeau and Satoshi Sekine. 2007. A survey of named entity recognition and classification. Lingvis- ticae Investigationes.
Thien Huu Nguyen, Kyunghyun Cho, and Ralph Grish- man, 2016. Joint event extraction via recurrent neural networks. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL).
I-Hung Hsu, Kuan-Hao Huang, Elizabeth Boschee, Scott Miller, Prem Natarajan, Kei-Wei Chang, and

---

Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic diffcrentiation in pytorch.
Hang Yang, Yubo Chen, Kang Liu, Yang Xiao, and Jun Zhao. 2018. DCFEE: A document-level Chinese financial event extraction system based on automat- ically labeled training data. In Proceedings of ACL 2018, System Demonstrations.
Yu Sun, Shuohuan Wang. Yukun Li, Shikun Feng, Xuyi Chen, Han Zhang, Xin Tian, Danxiang Zhu, Hao Tian, and Hua Wu. 2019. Ernie: Enhanced represen- tation through knowledge integration. arXiv preprint SSGiv:9904.0222
Hang Yang, Dianbo Sui, Yubo Chen, Kang Liu, Jun Zhao, and Taifeng Wang, 2021. Document-leve event extraction via parallel prediction networks. In Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguistics (ACL).
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in Neural Information Process- ing Systems (NeurIPS).
Sen Yang. Dawei Feng, Linbo Qiao, Zhigang Kan, and Dongsheng Li. 2019. Exploring pre-trained language models for event extraction and generation. In Pro- ceedings of the 57th Annual Meeting of the Associa- tion for Computational Linguistics (ACL).
Amir Pouran Ben Veyseh, Franck Dernoncourt, Quan Hung Tran, Varun Manjunatha, Lidan Wang, Rajiv Jain, Doo Soon Kim, Walter Chang, and Thien Huu Nguyen. 2021. Inducing rich interac- tion structures between words for document-lewe event argument extraction. Advances in Knowl- edge Discovery and Data Mining 25th Pacific-Asia Conference (PAKDD).
Zhisong Zhang, Xiang Kong, Zhengzhong Liu, Xuezhe Ma, and Eduard Hovy. 2020. A two-step approach for implicit event argument detection. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL).
Shun Zheng, Wei Cao, Wei Xu, and Jiang Bian. 2019. Doc2EDAG: An end-to-end document-level frame- work for Chinese financial event extraction. In Pro- ceedings of the 2019 Conference Empirical Meth- ods in Natural Language Processing (EMNLP).
Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015. Pointer networks. Advances in Neural Infor- mation Processing Systems (NeurIPS). 28.
Yang Zhou, Yubo Chen, Jun Zhao, Yin Wu, Jiexin Xu, and JinLong Li. 2021. What the role is vs. what plays the role: Semi-supervised event argument extraction via dual question answering. In Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI).
Kaiwen Wei, Xian Sun, 7equn 7hang. Jingyuan Zhang, Guo Zhi, and Li Jin. 2021. Trigger is not sufficient: Exploiting frame-aware knowledge for implicit event argument extraction. In Pruceedings of the 59th An- nual Meeting of the Association for Computational Linguistics (ACL).
Tong Zhu, Xiaoye Qu Wenliang Chen, Zhefeng Wang. Baoxing Huai, Nicholas Yuan, and Min Zhang. 2022. Efficient document- level event extraction via pseudo- trigger-aware pruned complete graph. In Proceed- ings of the Thirty-Firs, International Joint Confer- ence on Artificial Intelligence (IJCAI).
Xi Xiangyu, Wei Ye, Shikun Zhang, Quanxiu Wang, Huixing Jiang, and Wei Wu. 2021. Capturing event argument interaction via bi-directional entity-level recurrent decoder. In Proceedings of the 59th An- nual Meeting of the Association for Computational Linguistics (ACL).
A
Appendix
A.1 Index
Runxin Xu, Tianyu Liu, Lei Li, and Baobao Chang. event extraction via hetero-
2021a. Document- geneous graph-based based interaction model with tracker. In Proceedings of the 59th Annual Meeting of the As- sociation for Computational Linguistics (ACL).
We build an index of each appendix subsections in Table for better understanding our paper.
Subsection
Title
Appendix A.2
Symbol Reminder
Runxin Xu, Fuli Luo, Zhiyuan Zhang, Chuanqi Tan, Baobao Chang, Songfang Huang, and Fei Huang. 2021b. Raise a child in large language model: To- wards effective and generalizable fine-tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics.
Appendix A.3
Named Entity Recognition
Appendix A.4
Event Detection
Appendix A.
Two Stage Matching Function
Appendix A.6
Effect of Templates
Appendix A.7
Effect of Iteration Number
Appendix
Speed Comparison
Appendix
Study
Appendix Details Experiment Settings
Appendix A.
Details of Emitty Enhancement
Rumxin Xu, Peiyi Wang, Tianyu Liu, Shuang Zeng, Baobao Chang, and Zhifang Sui. 2022. two-stream AMR-enhanced model for document-level event ar- gument extraction. In Proceedings of the 2022 Con- ference of the North American Chapter of the Associ- ation for Computational Linguistics (NAACL).
Appendix A. 12
Details of Templates
Appendix A. 13
Details of EDAG Orders
Appendix A.14
Details of Optmmization Analysis
Table 6: The index table of appendix subsections.

---

A.2 Symbol Reminder
We obtain the candidate arguments representation H1o hiq.,h...... h& +1l E R[N++)jdd by max pooling corresponding entity mention rep- resentation, N₂ is the number of candidate argu- ments the document and h hx,+1 is an additional embedding to represent NULL argument. Simi- larly, we get the sentence representation H [hi, hg. ..... h RNN.xd by max pooling all word representations in the sentence.
We list the important symbols of methodology into Table 7, for better understanding our approach.
Symbol deN
Meaning
hidden dimension sentence number argument number role number template token number event record number ground truth record number iteration numher event type numher sentence representation argument representation event lype representation filled template representation event record representation event type score argument score for event roles event record filter output arguments (score) ground truth arguments (index) event record filter score ground truth record label
N, E N
Na N
N, N
N₂ e N
N₂ E
Then we make an interaction between sentence representation and argument representation by em- ploying a vanilla transformer encoder:
Ngt E N
NEN
NEN
II E E..d
H R.
[/1 11 Encoder2((ff,, (f1))
(17)
H E Roxd
II E Baxd
For focusing on the comparison event record generation module, we keep the same feature H c [NN++T)xd and H E IR Ne with (Zheng et al., 2019; Yang et al., 2021) in Equation (17).
II Rdied
P E R.N. P' (RRRAX PERR ET
Thus, use the argument-aware sentence representation H detect event types by multi-
Y" NNaXN
y P
head attention:
Y 1M
H Attention (Q", [H", H²), [H*, H²) (18)
Table The reminder of important symbols.
pe WW
(19)
where Q E R.Nexd We E Rd are trainable pa- rameters, 11 E Raxd is hidden representation, pe NR is the score of event types, and N, is the number of predefined event types.
A.3 Named Entity Recognition
N. that has N. sen- tences, IPGPF first embeds words of each sentence (w1, (w,, W...) from the document in- dividually, where Nw is the number of words in the sentence. Then these tokens are encoded by vanilla transformer encoder (Vaswani et al., 2017) to get the hidden representation:
Given a document D {
Finally, we train the modules for the event type detection task by optimizing the cross-entropy loss:
LED
I1yf 11)log log p
H EEnooderr((uu W2, .... WN..))
(15)
No I[ = 0) log (1 pf)
Where H" E Nuxd d is the hidden dimension. We further add a conditional random field (CRF) layer (Lafferty et al., 2001) after the encoder. Then we employ the BIO (Begin, Inside, Other) tag schema on word representation to extract entity mentions by maximum likelihood labeling:
where of {0, 13 is the i-th entry of golden label ye No and pf (0, is the i-th entry of po.
A.5 Two Stage Matching Function
Greedy Role Matching After getting the cost function refer to Equation
.NER = ΣΣ log P)ylw)
(16)
RED WEB
N, Yi log
(Y vD)
Σ A=1
where y is the label of the word
A.4 Event Detection
We greedily assign the most similar ground truth record for each Y? as its label:
After sentence-level named entity recognition, we get the entity mentions E = He where Ne is the number of entity mentions in the document.
j = arg min Crote (Y,',

---

have been generated in history, we assign la- bels from ground truth records without replace- ment. After all ground truth records have been assigned, i.e., all records the document have been discovered, we put all ground truth records back and greedily choose label with replacement. To inspire our model to generate records that
retain the special tokens corresponding event roles. Results in Tabel 8 show 1.5 FI decrease on multi-record set compared with IPGPF-remove context, which indicates template context is helpful for multiple events scenarios. Details of templates
without context can be found in Table 8.
Model
P
R
F1
S.
M.
Thus, the greedy many-to-one matching can be described surjective function (r(1) j, E
IPGPF
82.0
73.8
77.7
87.6
66.0
-caange context
-0.1
-0.2
-0.1
-0.7
+0.5
ド=,j E }²) mapping i-th oulput event record to j- th ground truth record, and satisfying vj 1 3i E Í (i) j.
-renove context
+1.3
-1.7
-0.4
+0.6
-1.6
IPGPF
85.7
77.3
81.3
91.0
70.1
-hange context
+0.4
-0.2
+0.1
-0,2
+0.4
-remove
+0.5
-1.5
-0.6
+0.3
-2.1
Filtered Record Matching After obtaining the total cost function refer to Equation 10:
Table 8: Performance of IPGPF for manual templates.
The change of template contexts causes little fluctuation. The removal of the template context reduces the Recall and multi-record set Fl Fl-score. S.: single-record set; M.: multi-record set.
Caul (Y, Yj) Cevent (ii 0)) 07)++ Croto(Y Yf) ,Y)
To find the best matching between output event records and ground truth records, we define an in- jective function {ū] = i.j E I E }*} that map index j from ground truth records index set Is index i of output records index set fa, and satisfying { (f() + x (1 # j2, j2 E I*). Denoting IT( /²) as the set all injection map-
A.7 Effect of Iteration Number
To investigate the effect of the iteration number of IPGPF. We list the model performance of different iteration numbers in Figure 6. Specially, we set the iteration number from {1, 7, 7,9, 11, 13, 15). As shown in Figure 6, after the iteration number larger than 5, the overall Fl score in the range from 77.5 to 77.7, and the FI score fluctuation on both single-record set and multi-record set are within 1 point. Even if the iteration number is 15, the model performance does not get significant decrease. The slight fluctuation during the increase of the iteration numbers suggests the robustness of our IPGPF model.
pings from Ia to I=, we choose the optimal map- ping by searching the minimal cost:
π
= === arg min
Σ Can(Pz() =c 1=1
(20)
the minimal cost mapping can be effectively com- puted by the hungarian algorithm (Kuhn, 1955).
Then we can compute the binary cross-entropy loss for event filtering:
90
Lreennt
log P
Σ log p()) (21) iEB
F1-score(%)
75
77.5
77.6
77.5
17.3
17.7
77.6
iEA
60
64.6
65.5
05.6
66.1
66.
Overall
where A x² (1=) and B f= (1=).
x
Single
Multi
30
A.6 Effect of Templates
5
9 11
13
15
Iteration Number
To explore the influence different template con- texts, we change the template contexts for each event type (IPGPF/PPPG -change context). As shown in Tabel 8, the change of template con- text just brings a slight performance fluctuation to IPGPF-change context compared with IPGPF Details of default templates and replacement templates are in Table 8.
Figure 6: The performance of IPGPF for different it- eration numbers. Overall: the entire ChFinAnn test set; Single: single-record set; Multi: multi-record set. The performance on all datasets improves rapidly as the number of iterations increases and stabilizes after the number of iterations is greater than 5.
A.8 Speed Comparison
To verify the effect of template context, we remove the contexts from templates (IPGPE/PFPG -remoee context) and just
We list the comparison training and inference speed Table 9. As auto-regressive generation

---

models, Doc2EDAG, GIT, and RAAT are the slow- est in both training and inference speed, whereas the efficient model PTPCG and the parallel model DE-PPN have faster speed, but their performance is relatively lower than other models.
to decrease the incorrect arguments than DE--PNN, still has the inadequate training and misses all arguments for TotalHoldingRatio. Fortunately, the usage of our pre-filling strategy alleviates the inad- equate training in parallel methods, and obtains the best results compared to other models. The only in- correct argument Jun 27, 2016 is not actual error, the reason is all pledge release records in ground truth just ReleasedDate, but set StartDate and EndDate be NULL.
In comparison, our IPGPF+ achieves significant better performance than the current state-of-the- art model RAAT while also having significantly faster training and inference speed. Additionally, we have found that sctting the iteration number of IPGPF/ IPGPF* to (default is 10) a viable op- tion that sacrifices a slight amount of performance for much faster training speed (16% of RAAT'S training time).
A.10 Details of Experiment Settings
Device and Data To ensure a fair comparison, we run all experiments on the same 8 Tesla- 100 GPUs under the same version of python dependen- cies. However, we had to use 8 Tesla-A GPUs to reproduce the RAAT model on the DuEE-fin dataset, as it uses too many features and causes an out-of-memory problem on V100 GPUs. All compared models in our experiments use the same train/dev/test split for both ChFinAmn and DuEE- fin
One Epoch Training (mins)
Total Training (days)
Inference Speed (docs/s
Overall
Model
DocZEDAG
15.8
2.2
13.2
55.5
DE-PPN
3.5
0.5
24.6
45.8
IPGPF
9.0
1.3
18.7
57.5
IPGPF-5
0.8
PTPCG
0.3
GIT
17.7
2.5
RAAT
44.6
6.2
62.8
IGPF+
11.3
1.6
11.6
64.8
|PGPF+55
7.2
1.0
14.9
64.5
Model Reproduction We reproduce the com- pared models Doc2EDAG DE-PPN $, PTPCG 6 GIT and RAAT by using the code they released, and use the same parameter settings in their paper. We use the results reported in the SCDEE paper since it is not open-sourced. We the results of DuEE-fin dataset reported in RAAT paper be- cause it just released the human selected order of ChlinAnn dataset but did not release the order for DuEF-fin dataset.
Table The comparison of training and inference speed between different models. We experiment with all mod- els on the DuEE-fin dataset by 8*32G Tesla V100 GPUs. IPGPF-5 IPGPF+-5 means the iteration number of IPGPF/ / IPGPF+ is 5 (default is 10).
A.9 Case Study
To demonstrate the difference between our itera- tively parallel method with auto-regressive meth- ods and parallel methods, we conduct a case study on the auto-regressive model Dec2EDAG, par- allel model DE-PPN and our iteratively parallel model IPGPF, which use the same NER and ED modules but different ERG modules. Figure 7 shows the prediction results of different generation methods. The document case contains two Equi- tyPledge event records, one is an equity pledge release, and another is an equity re-pledge. For auto-regressive models, after changing the genera- tion order of event roles, Doc2EDAG-RS- loses a whole event record than Doc2EDAG-HS, For par- allel models, DE-PPN failed to learn the event role TotdaPledgedShares and always generates NULL for CotalPledgedShares, which encounters zero- accuracy due to the inadequate training. After removing the pre-fillee filliin strategy, although IPGPF w/o pre-filling leverages the iterative generation
We note that DE-PPN employs extra training tricks and data augmentation, which are not ex- plicitly mentioned in their paper. In our reproduc- tion of DE-PPN, adopt the extra training tricks and achieve similar performance as described in their official response. However, we have refrained from using any extra training data, as such data
master/Data. /Data. zip
httpps:/aiistuioo baid:tuwddoo...uu....o..uu cot.aisttiioo///////o/o/o
competition//etaal//46
(commit da4f4hc)
/https://githhii http://ggttubb.ccco
(commit b2bc813)
(commit 3fe44b4)
"https://githut ptg.ttuub..com//uninn/////
(commit 3191743)
hittps://githann
EventExtraction RRAT
(commit 7a9a726)

---

Layers of Encoder2 are neither mentioned in the DE-PPN paper nor open-sourced. This decision was taken to maintain consistency with the experimental setup reported in the original paper and to ensure a fair comparison with other models that were not using extra training data.
pared models the DuEli-fi dataset since DuEE- lin has more event and argument types but less training data than the ChFinAnn dataset.
A.11 Details of Entity Enhancement
Table 11 lists our merge rule of entity labels on the ChFinAnn dataset. To further reduce the missing entity labels, we add additional entities matched through regular expressions (Zhu et al., 2022). Regarding the pre-trained encoder, we uti- lize LERT (Cui et al., 2022) for ChFinAnn and ERNIE (Sun et al., 2019) for DuFE-fin.
Hyperparameters
Value
Batch Size
64
Gradient Accumulation Steps
Training Liochs
200
Pre-filling Epochs
30
Learning Rate
3e-5 5c-5
Hidden Dimension
768
To make a fair comparison of different event generation types, IPGPF uses the same entity anno- tations and entity labels with Doc2EDAG, DE-PPN. We only merge entities or add additional entities for IPGPF when compared with the entity-enhance models such as SCDEE, PTPCG, GIT and RAAT.
Dropout Rate
0.1
0.5
g
0.75
m
0.5
1.0
A1
0.1
0.4
Aa
0.5
Kutity Type Holia
Simplified Type
Layers of Encoderl
4
Penson
Company Name Pladger
Layers Decoderl
Organization
Piedgee
Layers of Decoder2
4
Repurchise Amount
HishesTTadingPies
Table 10: Hyperparameters of our IPGPF model
Lowes(Iradingrrie
Amount
AveragePrice
StantJute
EadiDate
Hyperparameters To mitigate the error- propagation of entity mentions recognized by the model, we leverage the same scheduled sampling (Bengio et al., 2015) (Zheng et al., 2019) did. In the training process, we golden entity mentions from epoch epoch 10. From epoch 10 epoch 20, we gradually switch the mentions from golden label to model output results, by a linear decrease proportion from 100% to 0%. In our enhanced model !PGPF!, we use the ChildTuning (Xu et al., 2021b) to adapt the model to downstream tasks.
ReleasodDate
Date
ClosingDatte
UnfrozeDate
TotalHoldingSuurs
ToulPRolgedSSane
PiedgedShares
ShareS
Share
RepunchavvlSSuuess
TradedShanes
LateriHoldingSharss
Table 11: The entity label merge rule for the ChFinAnn dataset. Raw entity labels come from the event argument labels.
For the pre-filling strategy, linearly decrease the pre-filling probability from 120% to 0% in the first 30 epochs of model training, A will be set to 100% if バ > 100% at the current epoch.
A.12 Details of Templates
We listed the templates for the ChFinAnn dataset in Figure 8. As shown in Figure 8. In the comparison of different templates corresponding to Table 8, we use the default context to train our IPGPF model, the changed context to train the IPGPF-change con- text model, and the none context to train the IPGPF- remove context model. All templates for the model training on ChFinAnn are in Chinese, and here we translate these templates to English for illustration.
We implement IPGPF under Pytorch (Paszke et al., 2017) based codes released by (Zheng al., 2019) and (Yang et al., 2021).
For hyperparameters, we do not tune all the hy- perparameters in our experiment. For model com- parison, we use the same random seed following (Zheng et al., 2019) for compared models.
For the training epochs, we follow (Zheng et al., 2019; Yang et al., 2021; ; Xu al., 2021a) to train 100 epuchs for all compared models on the ChFi- nnnn dataset. We train 200 epochs for all com-
A.13 Details of EDAG Orders
As shown in Figure 12, we list the relation between the event roles in the ChFinAnn dataset and their

---

Sentence 10
Test
(83)
the 05,00000 sluns (8.33% oonjany shune cxpioil plodgst ICBC Taishen Beanch 2016. ZhongXai los plodged 2500000000 staces of the compuny's restricedd shares hold by laccouating for 8.39% the compuny's sotal share copital and 16.18% the companyss i staves) to ICBC Trizhos Branch for pesind From Sep 25, 2015 Sep. 17, 2021, the relevant plodge legistution proceduces have been completed.
On Sep 3818, Xin
[86]
(87)
As of annoncemens, ZhongKin
151,50 (97 shures
company. aocouniig 4815 the company' total shure capital.
[58]
The total nimber of plodged shares Zhonggin Iae 142,250000 sharcs, accouniing for 92 12.07% the company shaees and 47.30% of the compaty's lonal share capital
Event seconds of EquiyyPeedze (EP): Ground Truth
Roseed
Pledger
Plodgee
TeeaHHofingStuess
TesiHooliinRato
TotalPtolssSShees
SuriDatt
FadDatc
[1]]
ZiongXin Inc 2500000000 stores
ICBC Taiehou Branch
154, 308 407 shanes
$1.48%
140 202,0500000 stuers
NETL.
NULL
Sep 25, 2018
[82]
ZhongXin
25/0000000 stores
ICBC Tatchou Branch
155 shares
51,405
142,2501100 stores
25, 2018
Sep 1722
NULL
lovent reconds /lodge (LIP;: DecLEDAG-HS
Ressed
Pledger
PlaigyulSterss
Pledgre
TecaHHollinShumss
TesuHHollingRaiio
TotalPaalgsSShaess
StorDaate
FadMaatc
ReloreelDaate
IXI]
ZhongXin Inc
2500 stares
ICBC Taichou Branch
155,505, 297 shares
51,0016
NULL
Jun 2016
NULL
Sep 25, 2018
[R2]
ZhongXin
p50000,000 shores
ICBC Taichon Branch
154,500 shares
$1.48%
NULL
Sep 2018 Sep 1021
NULL
Even records of Eiadgy UPP;: Dee2LRAG-RS-1
Recoed
PledgedStarres
Pledgee
DonaHHodingShures
TonuHHoodngRato
TeudiPeegeddsurss
StarDuute
EndDuue
Releasediate
IRI|
ZhongXin Inc
2500000000 stares
ICBC Taichou Branch
154 308, 297 shares
51,48%6
250 stares
Sep 25, 2018
NULL
NULL
recomls EquuryYiedor (UP)::
Receed
Pledger
PloigealSSuers
Plotgee
TeeiHHollngSSurrs
TeeuHHolffuRRati
TouiPlolgstShhees
StortDate
FndDaai
RelecsedDate
[K]]
ZhongXin
25 (000000000
[CBC Taichou Branch
shares
51,08%
NULL
2018
NULL
NULL
[821
ZhongXin Ine
J000 slaues
ICBC Taizhoa Branch
354-500,.44 207 shares
51.48%
NULL
Jun 27, 2016 Sep 17, 1702
NULL
records EquityPledge (BP). IPGPE pre Ailin
Record ID
Pledger
PlesggedStures
Pledgee
TocillooiingShaes
StarDotte
Endiaate
Retessedlune
IRI|
ZhongXin
25,000/000
ICBC Taizhou Branch
154, (54,5508.40 ,97 shuses
NULL
142,250,000
2018
NULL
NULL
[821
ZhongXin Inc
2500000000 stases
ICBC Talzhou Branch
154,508, 297 shares
NULL
142,250 staues
Jun 27, 2016
Sep 1722
NULL
lvee secords of EquitsPledge (EF%
Record ID
Plodger
PlodgodStawes
Plodaee
ToaaHoodinghuurss
ToulHoolingRato
ToulPPedgedSluues
StunDute
EndDar
ReleasdDDaee
IRI|
ZhongXin
25,0000000
ICBC Taizhou Branch
154,508,497
51.48%
27, 2016
NULL
[22]
XiongXin Ine
25/000000 shoens
ICRC Toichou Branch
154,308 407 shanes
$1.48%
142 112,5000000000 stsers
Sep 25, 2018
Sep 17, 2001
NUIL
Figure 7: A Case for model comparison. We show the text of related sentences. Ground truth entity mentions and arguments are colored in blue, while incorrectly predicted arguments are colored in red. Event roles that get zero accuracies on the whole test are highlighted
ids. The event role generation orders of the auto- regressive EDAG models in experiments are listed in Table 13 to Table 18 by their ids.
The gradient norm ratio of query parameters cor- responding zero-accuracy event roles decreases around zero rapidly and fails escape, while the gradient norm ratio corresponding non-zero- accuracy event roles is still larger than zero sig- nificantly.
A.14 Details of Optimization Analysis
Figure 9, 10, and 11 show the gradient norm ratio of all event role queries of the parallel models DE- PPN, IPGPF-without pre-filling, and our IPGPF during the first 3, 000 training steps on the DuBE- fin training set which has 13 event types. In detail, each query is a 768-dimensional query tensor in the model parameters corresponding to an event role. Each broken line in the figure corresponds to one event role, the repeated line refers to the overlapped event roles cross different event types. To better show the comparison, we choose a value for the broken line every 100 steps in the first 3, 000 training steps, and draw the final line with a moving average of 5.
After using the pre-filling strategy, our IPGPF solves the zero-accuracy problem for almost all event roles. The reason for the rest 4 event roles with zero accuracies is the few role arguments in the DuEli-fin dataset. For instance, the event role "Underweighu/Holddng Ratio" just has 20 and 1 not NULL arguments on the DuEE-fin training set and dev set, respectively. Moreover, these 4 event roles get zero accuracies for all other compared models such as Doc2EDAG.

---

Remove Context (IPGPF-remmn -remove context)
Event Type
Template
Fquettyreesz
UnfroceDate Legallnstinuton FrogeShures EquityHolder SturDDatte EndDate TotalHoolingghhrrs TosaHHoiddgRatio..
EquittRepurcasse
CompanNaam RepucchasoSlooes
HishesfrodliggPiie
LowestrollingPiice ClosingDote Repurckuseckoount
PquiryHoldar TradedShares AveragePrice StorDoot FadDane LderHoidingghures
Oveneegee
EquiryHoliler TradedShares AverogePrice StorDooe EadDone
LaterHoidngghuees
Equittyledge
Relensedoote
Piedler PledgedShares Pledgee StortDote EndDote YoluHooiiinghhuee TolalHoldingRato TeloiPledgedSSooess
Default Context (IPGPE)
Event Type
Template
On UnfroneDute, Legulleatiution freeass or unfreeaes SrozeShures held by HqwiriHolder. starts StortDatt at DadeDate. At present, helcheiit still holds SonaHHoldinggharss shares of the company, accounting for TondHooldiogRatoo of the company's total share capital.
Equutty Preety
CompanyNamee repurchases RepurcharedSherer at the highest price of HighesfFradingPiis per share and the lowest price of Lowestrrddng per share, the repurchase
ReguityRepuchooee
ClosiegDote, and total payment amount about RepurchaeAwowat company the average price AveragePrice per share.
EquiryHolder sells TradedShares
starts from StariDote
Uguiityindeeeiggh
EndDore, Afler the sule, he'sheiit still holds Loter Shores shares of the company.
HquieyHolder buys PradeilSonss of the company the avengge price of Avengge per share. The purchase starts from StorDDote and cods a EndDare Afler the punchase, he/seri still holds LaterHoolingShere shares of the compuny.
EquittyCverweihht
On ReleaseDutte, Pledger pledges or relenses PledgedSSares to Pledgez. It starts from StoriDote and eads EndDDote Ho/Shelft still holds TotolHollinggaamm Shares of the company, accounting for TntalHoldingaatio of the company's 's total share capital, and accumalatively
Pludty
pladges Tone/PleyyeShhure shares. Changed Contest PPCPPr- context) Template
Event Type
From StareDhate to FniDDote FrogeShatet of EquityHolder were frozen or unfrozen by Legallnstiaation The unfrozen date Difhnnge At present, ho/sholit still holds TotolHoldingooio of the company's is total share capital which are TouulHoldingShores in total
Ereetyyreeze
On ClosineDute, RepurchosedSheve are repurchased by Compor-Nome The highest price is Hishee//rodingPiiee and the lowest price is Lowee/YrofiesPrice ReprochaseAmount paid for this reprchosement lotal.
Eqqutty Reparckee
From SturiDutt to EndDute. TrodedShares are sold by EmwirHobldrr at the average price of AveragePPice per share. After the sale, he/khefi still hold LaterHHoldinghuure shares of the company.
Unddeeiinne
From StariDate EndDere. TradedShores are bought by EquiryHoider at average price AveragePrice per Share..Aeee the purchase. he he/she/t still holds LaterHoodiinShhaee shares of the company.
EquiitOOverweihht
From SturtDute EndDute, PledgedSheres pledged released to Pledges by Pledeer. The release date RelewedDere, The cumalative shares for pladgement are ShaiPledgedSheme TomnHHoidwggotio of the company's 's total share capital were still held by heishofir, which together aecount for TotalHolding Shaves.
Pquittyledge
Figure 8: Details for templates, The expressions and event role orders differ between default and replacement templates. We further remove all contexts in templates to build the none-template.
DO / GIT- / /RAAT-HS
Event Role
ID
Event Type
Event Role Order
EquiyHolder
EquityFreezee
1++2 2-2-3300
+ 66-7-8
FroveShures
2
RopurthRpeeeeaee
9- 9-+10-11
122-18-144
LegalInstirution
3
Equityy Linderweight
1- 15 1- 7- 17
TotalHolldiggharess
Equity Overweight EquityPledge
17
TotaHHoldingRatio
18 18 20-21---77
StarlDate
EndDate
7
The event role generation orders for Dec2EDAG-O and GIT-O. Event roles are represented by their ids.
Table 13:
UnfroweDate
8
CompanyName
9
Highes(TudingPrie
10
Lowest SwwsstrradingPiss
11
RepurchasedShares
ClosingDate
RepurchuseAmmunt
14
TradedSbares
15
LaterHoldingSares
16
DoeZEDAG-RS RS-I/ CIL-L//II--S-IIII REEARSSIRRRAARRSSS 1
AveragePrice
17
Event Type
Event Role Order
Pledger
18
EquityFreezee
6-55-22
3-+3--77
11 11
PlesgeedShares
19
Underweidd
16-4 17
Pledgee
20
Overweiher
15-+1711-4 16 ----2---------------------------
VlotalPeeggedShhrress
21
EquityPeedge
ReleasedDate
22
Table 14: The event role generation orders for Doc2EDAG-1 and GIT-1. Event roles are represented by their ids.
Table 12: The relation between event roles in ChFinAnn and their ids.

---

Eevent Ipp:: Picdge]
[Event lype: Sherdttedempiion)
[Event type: Medgello cosd]
[Event Type: Admonition
1.0
Quaker Bar R21
0.41
1922
1950
2000
2000
301
1000
1993
2032
2500
300
1500
2000
2000
502
1000
1993
2032
2500
Taitrr
Thiniig Step
Tailig Sepp
|Event Type: emerrniookculsstnn
[Evont Type: Starchaldarte tree
[Event typc: ErecutieeChange
[Event Type: SidWin)
1.0
Oraten Rove Kate
Gradan Norm Rale
0.8
6.4
2000
2id0
1002 19d0
1903
Thining Step
Tuirrg Skp
Thining Step
Tuiiing Phip
[Evont CompanyUrting)
[Event CorporoteSincee)
[Event RnanceDetcct)
|Event Type: Sharcholdorkold
1.2
horm batto
0.3
2010
Type: Compny/aoirppery)
2000
Figure 9: Gradient Norm Ratio Analysis of DE-PPN Model on DuEE-fin Training Set. The plot depicts the gradient norm ratio of the queries associated with each event role in the DuEE-fin dataset during the initial 3,000 training steps. The solid line represents event roles with non-zero accuracy on the dev/test set, while the dashed line denotes event roles with zero accuracy.
DoeZEDAG-RS-/ -IT-RRS/SS/-IRII//// RAAT-RS-2
SAZEDAG-RS-IGGIT-RSS. -
Type
Role
Type
Order
EquikyFreeze
6-5544-- 1-5-4477
Equity
1+1-3-1 - 2 43-6-7
AquityRepurchan
14 13
10
12
SquityRepurchane
10 9
Undeewiindew
EquityUnnerwwein Underweight
Equity Overwwihttt
17 17-415-22 15-41-1----
11
Equity Overweiht EquityPledge
1 13-11--100 17-7
EquityPledge
7 12 2222222222222 22-1222222--
19 - - 1-7
+1
Table 15:
The event role generation orders for Doc2EDAG-2 and GIT-2. Event roles are represented by their ids.
Table 17:
The event role generation orders for Doc2EDAG-4 and GIT-4. Event roles are represented by their ids.
DeCIEDAG-RSA/ GIT-RS-3/ RAAT-RS-3
DeCEEDAG-RS-S/GTEE RIAIIRRS-S/RAAR SAIIS-S/RRAATEEER RAAT-RSS-
Event Type
Event Role Order
Event Type
Event Role Order
EquityFrocze
7 - 1-22-1
EquityFrozze
8-4
1+33-3 +1 + 2 -
Equity Repunchase
Equity Repurchee
Equity Underweight
17-+ 7-
EgpittUnndewweihh
17-
Overweihtt
7- 17
EquityOveeeiihtx
7 +7
EquityPledge
21 40-
EquityPledge
22 + 20
Table 16:
The event role generation orders for Doc2EDAG-3 and GIT-3, Event roles are represented by their ids.
Table 18:
The event role generation orders for Doc2EDAG-5 and GIT-5, Event roles are represented by their ids.

---

2000 [Event bpe Pledge]
[Evan: Typu: Sharshedemmpion)
[Evant Type: Hedgelt uase]
(Evant lppe Admonitinn)
Snadan FnadartWar Hore Bato
6.6
Guaken Mon
300
1500
2000
2000
993
1000
1907
2002
2500
300
100s
1500
2000
200
502
1000
1900
2002
2500
lEvent Tipe: Encerprised pnopcriookcusstto))
[Event Type: Sharchaldrtto dingneressc)
[Event Type: ExecutieChengel
[Event Type: BidWin)
Dustes Ber Rate
500
Loze
1500
2000
2000
092
1000
1000
2002
2000
500
Loos
2000
2000
902
1000
Li00
2002
Telnir Sep
Teinin Sep
[Event Type: CompanyUsting)
[Event Type: Sorporote
[Event Typc: FnanceDefiitt
[Event Type: Sharchilerriiid nnreree noDrraaase
1.4
Grdeetomm
0.
5c0
3500
2000
2000
593
1000
2002
2510
500
1058
3560
2000
2000
502
1000
1500
2022
3500
Traning Siep
Telrig Skp
Turin Siee
Tuorrng 7up
[Event Compan/tackrugery)
3.2
Figure 10: Gradient Norm Ratio Analysis of IPGPF-withore pre-filling Model DuEE-fin fin Training Set. The plot depicts the gradient norm ratio of the queries associated with each event role in the DuEE-fin fin dataset during the initial 3,000 training steps. The solid line represents event roles with non-zero accuracy on the dev/test set, while the dashed line denotes event roles with zero accuracy.
[Dvent Type: Piedge]
[tvent Type: Snarstaedemptionn
[Dvent Type: Pledgell
[Dvent Admontttonn
1.7
Stare
Taa
Talving
Baring Stre
Tnaving Stpp
Type:
tpp::
lype:
Chocke More Ry1
Grudare Nome haro
St
Pesker Morr
[Event Type: Companylisting)
Type: CorperateFinse)]
[Event Type: FnanceSefidd
[Event Type: chuccholdrrtddnnpprrrooeo
Grudan Now 3270
0.8
6.
Groken More
5C0
Los8
19t0
2000
2000
1000
1991
2500
25te
5tc
2000
2003
502
1100
1991
2550
[Evont Type: Compangkonkrugcc))
1.2
0.6
0.4
500
1000
2900
Thanhing
Figure 11: Gradient Norm Ratio Analysis of our IPGPF Model on DuEE-fin Training Set. The plot depicts the gradient norm ratio of the queries associated with each event role in the DuEE-fit fin dataset during the initial 3,000 training steps. The solid line represents event roles with non-zero accuracy the dev/test set, while the dashed line denotes event roles with zero accuracy.


Reviews:
[{"review_id": "539mxxYWGf", "reviewer": "Reviewer_wZ9b", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 1, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for his/her effort in reviewing our paper and his/her positive feedback. The reviewer brings forward constructive questions. In our response, we repeat each question in **bold** and write our answer follow the corresponding questions.\n\n## Q1(Weakness 1): The proposed method is complex and may be difficult to implement for researchers who are not familiar with the techniques used in this paper.\n\nThank you for bringing up this concern. We acknowledge that the proposed method in our paper involves multiple subtasks and may appear complex, especially for researchers who are not familiar with the specific techniques employed. In Section 2 of our paper, we highlight the challenges of the trigger-free doc-level event extraction (DEE) task, which requires extracting multiple event records from a document without annotated triggers. As a result, this task encompasses several subtasks, including named entity recognition (NER), event detection (ED), and event record generation (ERG). Both our proposed methods and previous related methods involve these three pipeline subtasks.\n\nHowever, we would like to emphasize that the core contribution of our paper, the pre-filling strategy, can have broader applications beyond the specific task discussed. As described in Section 3.2 of our paper, the pre-filling strategy pre-fills the correctly generated historical results to the current generated event record, and then fills the unfilled parts. Section 4.6.3 of our paper demonstrate this strategy effectively enhances parameter optimization during model training. We believe that the pre-filling strategy, or its variations, can be utilized in other parallel generation methods for various NLP tasks to improve the parameter optimization of model training. Exploring these possibilities is part of our future research agenda.\n\nWhile we acknowledge the complexity of the proposed method, we will make efforts to provide detailed explanations, clear guidelines, and possibly code implementations to facilitate its adoption by researchers who may not be familiar with the underlying techniques. We aim to ensure that our work is accessible and reproducible for the broader research community.\n\n## Q2(Weakness 2): The experiments are conducted on a limited number of datasets, which may not fully represent the diversity of real-world scenarios.\n\nWe appreciate the reviewer's comment regarding the diversity of real-world scenarios in our experiments. We would like to address this concern by providing the following clarification:\n\n1. The ChFinAnn dataset and the DuEE-fin dataset used in our experiments are both large-scale datasets. The ChFinAnn dataset consists of 32,040 documents with a total of 29,221 thousand tokens, making it one of the largest datasets in the field of Information Extraction. This dataset provides ample text and information for conducting a comprehensive analysis. Additionally, the DuEE-fin dataset contains 13 event types and a wide range of event roles (91), making it suitable for testing the performance of our model in diverse real-world scenarios.\n\n| Dataset | Event Types | Event Role Types | Document Number | Token Number |\n|---|---|---|---|---|\n| ChFinAnn | 5 | 35 | 32,040 | 29,221k |\n| DuEE-fin | 13 | 91 | 11,699 | 3,742k |\n\n2. These datasets were selected as they are the most suitable choices for our trigger-free document-level event extraction (DEE) research. We aimed to conduct a fair and comprehensive comparison between our proposed method and previous trigger-free DEE methods. Specifically, we followed the methodologies of previous related works, such as Doc2EDAG (2019, EMNLP), DE-PPN (2021, ACL), GIT (2021, ACL), and SCDEE (2021, EMNLP), using the ChFinAnn dataset. Additionally, we incorporated the recent works PTPCG (2022, IJCAI) and RAAT (2022, NAACL) and utilized the DuEE-fin dataset for further evaluation.\n\nFurthermore, it is important to note that conducting extensive experiments on large-scale datasets like ChFinAnn requires significant time and computational resources. As mentioned in the \"Limitations\" section of our paper and Appendix A.9 (Speed Comparison), training our proposed IPGPF model or other doc-level EE models used for comparison takes up to 150 hours per model using 8 * 32GB GPUs. This is due to the complexity of the doc-level EE task, which involves extracting numerous events from long documents.  Despite the challenges, we are committed to expanding our research by incorporating additional datasets in future work to further validate the effectiveness and applicability of our proposed method.\n\n## Q3(Weakness 3): The paper does not provide a detailed analysis of the errors made by the proposed method, which could limit the interpretability of the results.\n\nThank you for raising this concern. In order to provide a detailed analysis of the errors made by our proposed method, we have categorized them into three distinct types: selecting errors, missing errors, and filling errors. Selecting errors occur when the ground truth argument is A, but the model predicts argument B. Missing errors arise when the ground truth argument is A, but the model predicts \"NULL\" (indicating that the document does not mention any arguments for the corresponding event role). Filling errors, on the other hand, occur when the ground truth argument is \"NULL,\" but the model predicts argument A.\n\nTo illustrate these error types, we present an example in the \"EquityFreeze\" event type in the following table.\n\n| Event Role         | Ground Truth       | Prediction         | Error Types     |\n|--------------------|--------------------|--------------------|-----------------|\n| EquityHolder       | Jinggong Group     | Jinggong Group     | -               |\n| TradedShares       | 35,000 shares      | 100,000 shares     | selecting error |\n| StartDate          | Jan 13, 2019       | NULL               | missing error   |\n| EndDate            | NULL               | Jan 13, 2019       | filling error   |\n| LaterHoldingShares | 155,335,000 shares | 155,335,000 shares | -               |\n| AveragePrice       | 19.88              | 19.88              | -               |\n\nFurthermore, to quantify the occurrence of these errors, we randomly selected 50 documents from the ChFinAnn test set that contain incorrectly predicted event arguments. We then analyzed the proportions of the three error types. As shown in the table, we found that the primary error made by our method is the missing error, where the model fails to predict any arguments for the event role and instead predicts \"None.\" This can be attributed to the inherent difficulty of extracting all relevant information from long texts in doc-level event extraction (DEE) models. It is worth noting that the documents in the ChFinAnn dataset have an average length of 912 tokens, which is longer than most NLP datasets. In our future work, we are committed to enhancing the long-dependency modeling capacity of DEE models to mitigate the occurrence of missing errors.\n\n| Error Types | Proportion |\n|---|---|\n| selecting error | 26% |\n| missing error | 66% |\n| filling error | 8% |\n\n## Q4 (Risk 1): The complexity of the proposed method may make it difficult for some researchers to understand and replicate the results, which could limit the impact of the paper.\n\nThank you for your suggestion. In response to your feedback, we have provided an explanation for the complexity of the trigger-free document event extraction task and the rationale behind our proposed method. We would also like to highlight that our pre-filling strategy, along with its variants, has the potential to be applied to other parallel generation methods in various NLP tasks. For a more comprehensive understanding of these aspects, we kindly direct you to our detailed response to **Q1**.\n\n## Q5 (Risk 2): The limited number of datasets used in the experiments may raise questions about the generalizability of the results to other scenarios.\n\nThank you for your suggestion. In response to your feedback, we have provided a comprehensive interpretation to justify the suitability of the two public large-scale datasets we utilized for conducting a convincing analysis. For a more detailed explanation on this matter, we kindly refer you to our response to **Q2**.\n\n## Q6 (Risk 3): The lack of detailed error analysis may limit the interpretability of the results and make it difficult to identify areas for improvement.\n\nThank you for your suggestion. In response to your feedback, we have conducted a detailed analysis of the errors made by our proposed method. For a comprehensive understanding of the specific error types and their implications, we kindly direct you to our response to **Q3**.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "accepted into Findings?\nThe strengths of this paper include:\n\n1. The proposed IPGPF method achieves state-of-the-art performance on several datasets, outperforming previous models and auto-regressive models.\n2. The use of a pre-filling strategy and PT-PCG representation for events improves the efficiency and accuracy of event extraction.\n3. The experiments are conducted on two public datasets, which makes the results more generalizable and applicable to real-world scenarios.\n\nIf this paper were to be presented at the conference or accepted into Findings, the main benefits to the NLP community would be:\n\n1. The introduction of a new method for document-level event extraction that improves accuracy and efficiency, which could be applied to many natural language processing tasks.\n2. The use of a pre-filling strategy and PT-PCG representation for events could inspire new research directions and techniques for event extraction.\n3. The experiments and results provide a benchmark for future research on document-level event extraction, and could help researchers compare their methods to the state-of-the-art.", "reasons_to_reject": "The weaknesses of this paper include:\n\n1. The proposed method is complex and may be difficult to implement for researchers who are not familiar with the techniques used in this paper.\n2. The experiments are conducted on a limited number of datasets, which may not fully represent the diversity of real-world scenarios.\n3. The paper does not provide a detailed analysis of the errors made by the proposed method, which could limit the interpretability of the results.\n\nThe main risks of having this paper presented at the conference or accepted into Findings could include:\n\n1. The complexity of the proposed method may make it difficult for some researchers to understand and replicate the results, which could limit the impact of the paper.\n2. The limited number of datasets used in the experiments may raise questions about the generalizability of the results to other scenarios.\n3. The lack of detailed error analysis may limit the interpretability of the results and make it difficult to identify areas for improvement.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes a new method for document-level event extraction called the Iteratively Parallel Generation method with the Pre-Filling strategy (IPGPF). The problem that this paper addresses is the challenge of accurately extracting events from documents, which is important for many natural language processing tasks such as information retrieval, question answering, and summarization. \n\nThe main contributions of this paper are:\n\n1. The development of the IPGPF method, which outperforms previous models and achieves state-of-the-art performance on several datasets.\n2. The introduction of a new pseudo-trigger-aware pruned complete graph (PT-PCG) representation for events, which reduces the computational complexity of event extraction.\n3. The use of a pre-filling strategy to improve the efficiency and accuracy of event extraction, by generating candidate events before filling in the event arguments.\n\nOverall, this paper makes significant contributions to the field of document-level event extraction by proposing a new method that improves accuracy and efficiency, and by introducing a new representation for events that reduces computational complexity.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "HNfxPT033D", "reviewer": "Reviewer_wn1f", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for his/her effort in reviewing our paper and his/her positive feedback. The reviewer brings forward constructive questions. In our response, we repeat each question in **bold** and write our answer follow the corresponding questions.\n\n## Q1: Doesn’t talk about runtime implications for additional steps\n\nThank you for bringing up this concern. **As a supplement, we conducted experiments on the DuEE-fin dataset to examine the runtime implications of our proposed pre-filling strategy, two-stage matching, and the template.**  As shown in the following table, these modules do not significantly impact the training speed or inference speed, as their runtime variances are all below 5% for both training and inference. (The pre-filling strategy does not affect the inference speed since it is only utilized during training. A more detailed analysis can be found in our response to **Q2**.) \n\nOn the other hand, we found that removing these modules from our model leads to a significant decrease in overall performance. Therefore, we argue that maintaining these modules is crucial for achieving high-quality results. However, we acknowledge the importance of addressing runtime concerns and propose an alternative approach to improve training and inference speed without sacrificing performance.\n\nOur proposed iteratively parallel method allows for a reduction in the iteration number, resulting in a substantial acceleration of both training and inference processes, while only sacrificing a slight amount of performance. As shown in the following table, by setting the iteration number to 5 (default is 10), our IPGPF-5/IPGPF$^+$ models achieved more than a 30% reduction in training time and a substantial inference speed boost compared to the vanilla IPGPF/IPGPF$^+$. We provide a detailed analysis of the iteration number in Appendix A.8 of our paper.\n\nAdditionally, we have included a comprehensive analysis of the runtime/speed comparison between our proposed model and other related models in Appendix A.9 of our paper. This analysis offers a broader perspective on the runtime implications of our approach and demonstrates its better performance in comparison to existing methods.\n\n| Model | One Epoch Training (mins) | Total Training (days) | Inference Speed (docs/s) | Overall F1 |\n|---|:---:|:---:|:---:|---|\n| IPGPF | 9.0 | 1.3 | 18.7 | 57.5 |\n| - w/o pre-filling | -4.4% | -4.6% | +0.0% | -14.6 |\n| - w/o matching | -1.1% | -0.8% | +0.5% | -1.5 |\n| - w/o template | -2.2% | -2.3% | +1.1% | -0.3 |\n| IPGPF-5 | -37.8% | -38.5% | +35.3% | -0.2 |\n|  |  |  |  |  |\n| IPGPF$^+$ | 11.3 | 1.6 | 11.6 | 64.8 |\n| - w/o pre-filling | -4.4% | -4.4% | +0.0% | -25.8 |\n| - w/o matching | -0.9% | -0.6% | +0.9% | -2.3 |\n| - w/o template | -3.5% | -3.8% | +2.6% | -0.4 |\n| IPGPF$^+$-5 | -36.3% | -37.5% | +28.4% | -0.3 |\n\n## Q2: Preflling seems to require extensive knowledge of role types, which seems a limiting factor scaling\n\nWe want to clarify that our pre-filling strategy does not rely on extra role type information. Specifically, as mentioned in Section 3.2 in our paper, the pre-filling strategy pre-fills the correctly generated historical results to the current generated event record, and then fills the unfilled parts. The specification of event role positions in our paper is solely for the purpose of using artificial templates to enhance context understanding . It is important to note that these templates can be removed, and event role positions can be in any order. We have conducted corresponding experiments, which are reported in Appendix A.7, Table 8, and Figure 8 in our paper.\n\nIn fact, the only information required for the pre-filling strategy is the mapping between predicted arguments and ground truth arguments (just for training, not needed for inference). The mapping is used to determine whether the model has correctly predicted the arguments. This mapping is required not only for our pre-filling strategy but also for any other model needing to compute loss value during their training. Therefore, the pre-filling strategy, or its variations, can be employed in other parallel generation methods for NLP tasks to enhance the parameter optimization of model training. We intend to explore these possibilities in our future research.\n\nFurthermore, we would like to emphasize that our pre-filling strategy is primarily utilized to improve the optimization of model training, as mentioned in lines 296 to 302 of our paper. During inference, pre-filling is not used since the model has already been trained. Hence, we only use ground truth records to select the correct arguments for pre-filling during model training and do not require any ground truth results for inference. Additionally, to eliminate the risk of data leakage, we conducted experiments on the DuEE-fin online test set (Section 4 of our paper). The model's performance on this test set is evaluated online using a large-scale confusion set consisting of 55,881 documents. This approach effectively prevents data leakage and manual intervention, ensuring the reliability of our results.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* Well-written with a specific problem scope\n* Combines a number of strategies for the end to end task, introducing the pre-filling strategy among tweaking other parts\n* Has detailed explanation of the subtasks and methods\n* Compares against handful of recent baselines, performance not cleanly better on either P /R but overall improved F1\n", "reasons_to_reject": "* Doesn’t talk about runtime implications for additional steps\n* Preflling seems to require extensive knowledge of role types, which seems a limiting factor scaling", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper presents a method to address the problem of extracting multiple event records iteratively, while extracting event roles in parallel with different order of event roles by introducing a new template pre-filling strategy.\n", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "OdXOGspYcQ", "reviewer": "Reviewer_7svK", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are very grateful to the reviewer for his/her effort in reviewing our paper and his/her positive feedback. The strengths of our work as written by the reviewer are precise. Here below, we address every question and comment raised by the reviewer. We repeat each question in bold in our response and write our answers following the questions.\n## Q1: The fonts in figures are quite small for reading.\nWe would like to express our gratitude to the reviewer for bringing up this concern. We acknowledge that the fonts in the figures were relatively small in the initial version of our paper. Our intention was to accommodate additional crucial experiments within the main document to offer a comprehensive analysis of our method. However, **we understand the importance of readability and will address this issue in our revision by enlarging the font in the figures**. We sincerely appreciate the reviewer's time and effort in providing us with valuable feedback.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1 - The performance of proposed method is quite well on selected benchmarks.\n2 - The experiment parts are quite sufficient.\n3 – The focused problem “inadequate training” of parallel method is an important issue.", "reasons_to_reject": "1 – The fonts in figures are quite small for reading.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper is about a new parallel method IPGPF designed for Document-level Event Extraction (DEE). Authors propose pre-filling strategy to improve inadequate training problem, and propose iterative generation to utilize event records extracted previously. Main experimental results on ChFinAnn and DuEE-fin show that IPGPF reaches new SOTA performance. Other analysis experiments give sufficient views of IPGPF’s advantages.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper is about a new parallel method IPGPF designed for Document-level Event Extraction (DEE). They introduces a pseudo-trigger-aware pruned complete graph (PT-PCG) representation for events, which reduces the computational complexity of event extraction. They also use of a pre-filling strategy to improve the efficiency and accuracy of event extraction by generating candidate events before filling in the event arguments. Experimental results on ChFinAnn and DuEE-fin show that IPGPF reaches new SOTA performance. The main concern about this paper is the limited number and language of used evaluation datasets.
Decision: Accept-Main