CorefPrompt: Prompt-based Event Coreference Resolution by Measuring Event Type and Argument Compatibilities
Authors: Sheng Xu
PEIFENG LI
Qiaoming Zhu
Venue: EMNLP 2023 Main (2023)
Keywords: event coreference
event coreference resolution
prompt
TL;DR: We propose a prompt-based approach to transform event coreference resolution into a cloze-style masked language model task.

Abstract: Event coreference resolution (ECR) aims to group event mentions referring to the same real-world event into clusters. Most previous studies adopt the "encoding first, then scoring" framework, making the coreference judgment rely on event encoding. Furthermore, current methods struggle to leverage human-summarized ECR rules, e.g., coreferential events should have the same event type, to guide the model. To address these two issues, we propose a prompt-based approach, CorefPrompt, to transform ECR into a cloze-style MLM (masked language model) task. This allows for simultaneous event modeling and coreference discrimination within a single template, with a fully shared context. In addition, we introduce two auxiliary prompt tasks, event-type compatibility and argument compatibility, to explicitly demonstrate the reasoning process of ECR, which helps the model make final predictions. Experimental results show that our method CorefPrompt performs well in a state-of-the-art (SOTA) benchmark.

CorefPrompt:
Prompt-based Event Coreference Resolution by Measuring Event Type and Argument Compatibilities
Sheng Xu, Peifeng Li and Qiaoming Zhu
School of Computer Science and Technology, Soochow University, China sxu@stu. suda. edu. cn, {pfli, qmzhu)@suud. http://sddaauuuuu...ooo
Abstract
This example contains seven event mentions (ev1-e07) and ten entity mentions (arg1-ar910 that serve as arguments. Among them, the death event mention ev, with the argument argi and the death event mention ev;, with the arguments args. args- and argr are coreferential, as both of them describe the girl's suicide by jumping off building; the in- jury event mention et3 with the arguments arge and arga and the injury event ev4 with the argu- ment arg4 are coreferential, as both of them de- scribe the girl's disfigurement; other event men- tions are singletons. Identifying the coreference be- tween event mentions is essential for understanding the content of the text, and is the key to aggregat- ing information that can support many downstream NLP applications, such as event extraction (Huang and Peng, 2021), discourse analysis (Lee al., 2020) and timeline summarization (Li al., 2021).
Event coreference resolution (EBCR aims to group event mentions referring the same real- world event into clusters. Most previous studies adopt the "encoding first, then scoring" frame- work, making the coreference judgment rely on event encoding. Furthermore, current methods struggle to leverage -ummann-mmmrrizeed ECR rules, e.g., coreferential events should have the same event type, guide the model. To address these two issues, we propose prompt-based approach, CorefPrompt, to transform ECR into a cloze-style MLM (masked language model) task. This allows for simultaneous event mod- eling and coreference discrimination within single template, with fully shared context. In addition, we introduce two auxiliary prompt tasks, event-type compatibility and argument compatibility, explicitly demonstrate the rea- soning process of ECR, which helps the model make final predictions. Experimental results show that our method Core/Prompt performs well in a state-of of-the-art (SOTA) benchmark.
Event coreference resolution is more challenging than entity coreference resolution due the com- plex event structure (Yang et al., 2015), triggers and corresponding arguments are loosely distributed in the text, which needs to consider the compatibilities of multiple event dimensions, including triggers, arguments, event types, etc. Most previous work regards ECR as an event-pair classification task (Huang et al., 2019; Lu and Ng. 2021a; Xu et al., 2022), i.e., judging whether event mentions coreferential. Early neural methods focus on ob- taining trigger representations by various encoders and then manually constructing matching features (Krause et al., 2016; Nguyen et al., 2016), while recent studies integrate event compatibility into judgments using well-designed model structures (Huang et al., 2019; Lai et al., 2021; Lu and Ng, 202 la) directly incorporate argument informa- tion into event modeling (Zeng et al., 2020; Tran et al., 2021), alleviating noise brought by wrongly extracted or empty event slots. Other work (Kriman and Ji, 2021; Xu et al., 2022) learns ECR-aware event representations through contrastive learning
1 Introduction
Within-document event coreference resolution (ECR) task aims cluster event mentions (i.e., trig- gers that most clearly indicate the occurrences of events) a document such that all event mentions in the same cluster refer to the same real-world event. Consider the following example:
[Former Pakistani dancing girl) commits { (suicide)ev: 12 years after horrific (acid) [attack) eny which (left) (her)argy looking "not human".. (She)arga had undergone 39 sep- arate surgeries to repair [damage]ess Jex4- Leapt to (her) (death)evs from (sixth floor Rome building)arge (earlier this month)args (Her ex- husband)arra argg was [charged)cws ev, with [attempted murder /argu in (2002) Jargoo but has since been (acquitted)e..-
'Code is available at https: : :/ thub. wwwphuu....o//ss.eeeeee.eeee com/jsssx3360/ prampt-event worrff-enlp222.

---

multi-level modeling. However, least two limitations exist in the above studies.
captured features focus on other aspects and fail to discover the association between compatibili- ties and coreference due to the bias of interaction features or noise in the data. In this paper, we intro- duce two auxiliary prompt tasks, event-type com- patibility and argument compatibility, to explicitly demonstrate the inference process of coreference judgment in the template and guide the model to make final decisions based these compatibilities. Benefiting from prompting, our method can navi- gate the model's focus on event type and argument compatibilities using templates in natural language and can be adjusted to different compatibility levels with the assistance of soft label words.
First, most current ECR studies adopt the "en- coding first, then scoring" framework, wherein they first use encoders (e.g., BERT) to encode the text and obtain event mention embeddings, and then apply a scorer to evaluate the coreference score of event pairs based on these learned embeddings. This results in the "information blocking" issue. Es- sentially, since the scorer solely utilizes the learned embeddings as inputs, almost the entire corefer- ence determination relies on the event encoding, However, the event encoding performed inde- pendently, without direct influence from corefer- ence judgmeen.. As a result, the encoder may not accurately capture contextual information that is crucial for ECR. Especially for coreferential event pairs with unbalanced information, the learned em- beddings may significantly differ if one event has rich arguments while the other has only vague trigger. To alleviate this problem, Kenyon-Dean et al. (2018) and Kriman and Ji (2021) constrain event modeling by attraction and repulsion losses, making coreferential events have similar represen- tations. However, this approach still performs event modeling and coreference discrimination sep- arately, and Xu et al. (2022) find that uppropriate tensor matching can achieve similar effects. Obvi- ously, event modeling and coreference judgment are closely associated, and even humans need to review the original text to capture detailed clues when judging coreference. To address this issue, we convert ECR into a mask language prediction task using well-designed prompt, thus the simulta- neously performed event modeling and coreference judgment can interact conveniently based on fully shared context improve each other.
We summarize our contributions as follows:
Our prompt-based method CorefPrompt trans- forms ECR into an MLM task to model events and judge coreference simultaneously;
We introduce two auxiliary prompt tasks, event-type compatibility and argument com- patibility, to explicitly demonstrate the reason- ing process of ECR.
2 Related Work
Event Coreference Resolution Event coreference resolution is a crucial information extraction (IE) task. Except for few studies applying clustering methods (Chen and Ji, 2009; Peng et al., 2016). most researchers regard ECR as an event-pair clas- sification problem and focus on improving event representations. Due the complex event struc- ture, ECR needs to consider the compatibilities of multiple event dimensions, including triggers, arguments, event types, etc. Early work builds lin- guistical matching features via feature engincering (Chen et al., 2009; Liu et al., 2014; Krause et al., 2016), while recent studies incorporate element compatibility into discrimination softly by design- ing specific model structures (Huang al., 2019; Lai et al., 2021; Lu and Ng, 2021a,b) or enhancing event representations (Zeng et al., 2020; Tran et al., 2021). In particular, Huang et al. (2019) first train an argument compatibility scorer and then trans- fer it to ECR. Tran et al. (2021) build document graph structures and enhance event embeddings by capturing interactions among triggers, entities, and context words. Other methods learn ECR-aware event representations through contrastive learning or multi-level modeling (Kriman and Ji, 2021; Xu et al., 2022). For example, Xu et al. (2022) intro- duce full-text encoding and an event topic model
Second, previous methods struggle utilize hu- man knowledge, e.g., coreferential events should have the same event type and compatible argu- ments, and often require designing unique model structures to guide the model to focus on the com- patibility of event elements, e.g., strictly match- ing extracted event elements (Chen et al., 2009; Cybulska and Vossen, 2015) or softly integrating event compatibilities (Huang et al., 2019; Lai et al., 2021). These approaches rely on large amount of training, and cannot guarantee that the model can finally capture the compatibility features or under- stand the association between compatibilitics and coreference. In the worst cases, the automatically

---

learn document-level and topic-level event repre- sentations. However, they model events and judge coreference separately, and cannot leverage rich numan-summarizer rules to guide the prediction. Our approach transforms ECR into a mask lan- guage prediction task, simultancously performing event modeling and coreference judgment on shared context and guiding the model by explic- itly showing the inference process in the template. Prompt-based Methods for IE Recently, prompt- based methods (Brown et al., 2020) that convert tasks into form close to the pre-trained tasks have attracted considerable attention. Many re- cent IE studies (Lu et al., 2022) explore prompt- based methods, including named entity recogni- tion (Cui et al., 2021), relation extraction (Chia al., 2022; Chen et al., 2022; Yang and Song, 2022), event extraction (Li et al., 2022; Ma et al., 2022; Dai et al., 2022), and event causality identi- fication (Shen et al., 2022). These methods either transform the task into a cloze-style MLM problem or treat it as generative problem. In particular, Chen et al. (2022) inject knowledge into the prompt by constructing learnable virtual tokens contain- ing label semantics. Hsu al. (2022) construct prompt containing rich event information and then parse the elements. However, except for Shen et al. (2022) exploring event causality recognition rely- ing on logical reasoning, most studies directly lever- age PTM OPre-Traine Model) knowledge using schema. Therefore, constructing prompts for com- plex IE tasks remains an under-researched problem.
the mapping that converts category labels label words. Specifically, for each sample æ, prompt- based methods first construct an exclusive template (=)) containing a [MASK] token, and then send it into a PTM M to predict the score of each label word filling the [MASK] token, and calculate the probability of the corresponding category p(y)z))
p(y]x) p P[[MASK] v()]T(())
exp (M(((T)(((T)
(L)
EE exp PM (M((T((
where PM (4T(()) represents the confidence score of t token that fills the mask position in the template T (ax), predicted by the MLM head of M. In this paper, adopt prompting to solve the two issues mentioned before by designing specific templates for ECR. Specifically, for the input event mention pair (co,, cvy), we construct three corre- sponding templates: the prefix template Tpre. the anchor template Tane. and the inference template Tmn. These templates respectively add guidance, encode events (including predicting event types), and discriminate coreference (including judging type and argument compatibilities). Then, em- bed these templates into the segment hosting the two event mentions, converting the sample into a prompt containing multiple [MASK] tokens. Since all event tasks are completed simultancously in the same template, multiple steps, such as event mod- eling and coreference judgment, can interact con- veniently based on fully shared context, reducing the "information blocking" problem that existed in previous work. Finally, send the entire prompt to a PTM and use the PTM's ' MLM head obtain the results of all tasks by predicting mask tokens. We guide PTM encoding with the prefix template and explicitly demonstrate the reasoning process of coreference judgment with the inference tem- plate, fully incorporating human knowledge into the model predictions. The overall framework of our CorefPrompt is shown in Figure 1.
3 Model
Formally, given sample æ (D, (cvs, euy)). where D is a document and (cv;, evy) is an event mention pair in D. An ECR model first needs to judge the label y E Y [Coref, Non-Coref) for every event mention pair, and then organize all the events in D into clusters according to the predic- tions. Therefore, the key lies in the coreference classification of event mention pairs. Different from fine-tuning methods that classify based on learned event mention embeddings, the prompt- based methods we adopt reformulate the problem mask language modeling task.
3.2 ECR-aware Sequence Encoding
Considering that ECR is complex task requiring reasoning, we construct our prompt by mixing mul- tiple templates, similar to PTR (Han et al., 2022). First, following the common practice of prompting. we add prefix template Tyre that contains knowl- edge before the input text to guide the model:
3.1 Prompt-based method for ECR
The most popular prompting to convert classi- fication into a cloze-style MLM task using tem- plate T.)) and verbalizer v, where v Y + Vy is
Tpre: In the following text, the focus on the events expressed by [E1S] en [E1E] and [E2S]

---

Prefix Template:
".., the focus is on the events expressed by [E15] etj
[1E and [E2S] evy [E2E] and
Anchor Template:
"Here [E15] (Here(E15] eV;
[11E
expresses
[MASK] event with.. participants
Inference Template:
E15 ev:
[1EE
and [25]
foj
[2E have [MASK] event type and [MASK] participants ... refer to [MASK]
event."
Type
Event Type
Type Compatibility
Compatibility
Coreference
attack
injure
incompatible
compatible
Non-Coref
CorefPrompt
Head
MLM Head
MLM Head
MLM Head
MLM Head
fuse
Matching
fuse
fuse
Matching
[CLS) Prefixt..... exj
[15
[ELE]
[MAK
"mp e25
[E2E]
[MASK]
[MAS event type ...SSK
participants
MASK
SEP)
Prefix Template Tpre
Anchor Template Tane
Anchor Template Tane
Inference Template Finf
Figure I: The overall framework our prompt-based method Core/Prompt. We first utilize a prefix template Tpre to inform PTM what focus when encoding, then mark event types and arguments inserting anchor templates Tane around event mentions, and finally demonstrate the reasoning process of ECR using an inference template Tay which introduces two auxiliary prompt tasks, event-type compatibility and argument compatibility.
enj [E2E], and it needs to judge whether they refer to the same or different events.
word from the vocabulary as its label word. Thus, we create virtual label words for event types us- ing the semantical verbalizer (Chen et al., 2022; Dai al., 2022). Specifically, for each event type 8, we take its tokenized sequence { 2 gm as the semantic description, and then initialize the embedding of the corresponding label word I, as:
where [E1S]/[E1E] and [E2S] [CEE]] are special event markers to represent the start/end of two event triggers eu; and edg. which will then be added the vocabulary as learnable tokens. The prefix template first asks the model to spend additional attention on the two triggers to be processed when encoding the entire input, and then informs the model of the final objective to help the model learn embeddings relevant the ECR task.
m
E(L)
E(q
m 1=1
where E(-) represents the word embedding table. and then we expand the MLM head of PTM using these label words. During training, we calculate the cross-entropy loss L. between the predicted prob- ability distribution on label words and annotated event types as the supervision signal. In the pre- diction stage, we just need to keep these positions as [MASK]s. The anchor templates explicitly mark the event types and arguments and inserted near the event mention. Therefore, benefiting from the Attention mechanism, PTM can focus more on the argument information when encoding the context to support subsequent prompt tasks.
Previous work (Huang et al., 2019; Otmazgin al., 2022) shows that argument compatibility and mention type are key cues to judging corefer- ence. Therefore, we insert templates containing this information around event mentions, raising the model's attention to arguments and event types. Since these inserted templates can be seen as mark- ing the event positions in the document, we call them anchor templates Tane- Specifically, let the argument set of event mention ev; be A P'UL', where p (pi.p?....) L' 14,,2,..)) are the participant set and location set, respectively. The corresponding anchor template (the anchor tem-
plate form of EUj is same as that of ev;) is:
3.3 Joint Reasoning with Auxiliary Tasks
Tanc: Here [E1S] ev; [E1E] expresses [MASK] participants at 15,
After guiding the model to spend more attention on the event mention pair (ev,, ety) using the prefix and anchor templates, the easiest way finish ECR
event with p p2,
Here, introduce a derived prompt task pre- dict event types without inputting the recognized ones. Given that an event type may contain multi- ple tokens, it is not easy to directly find a suitable
concatenate template TECR. as shown follows, that converts the task into a mask language prediction problem and chooses

---

Vy {same, different) as the label word set. In this way, we can obtain the results by comparing the predicted confidence scores of these label words at the mask position.
combine additional clues predict tokens at mask positions in the inference template Tinf- Specifically, similar to traditional fine-tuning methods, we first apply the attention mechanism on top of the hidden vectors h2 of the j-th tokens in anchor template Tanc to obtain the embedding e; of event mention as follows:
TECR: In the above text, events expressed by [E15] en; [E1E] and [E2S] euj [E2E] refer to [MASK] event.
However, since the complex ECR task requires the model to measure the compatibility of multiple event dimensions, it is quite difficult for the PTM to predict coreference merely by encoding the content in the prompt. Inspired by Chain of thought (CoT) prompting (Wei et al., 2022; Qiao et al., 2022), which guides the model to make the final prediction by demonstrating the reasoning process step by step, we construct an inference template Tasf to explicitly show the reasoning process of ECR by introducing two auxiliary prompt tasks:
6
(3)
5=p
exp(w: (F)
04 L exp(w;)
Wi = w hi
(4)
where p and & the start and end positions of the event mention in the anchor template Tonc, re- spectively, and w. is the model parameter. Then, following Xu et al. (2022), we choose the element- wise product and multi-perspective cosine simi- larity MultiCos(.) as matching operations to cap- ture the semantic interactions on event triggers and event types:
Tinf: In conclusion, the events expressed by [E1S] ev [E1E] and [E2S] enyy [E2E] have [MASK] event type and [MASK] participants, so they refer to [MASK] event.
M(x -ax1 o x2; MultiCos(z1, x22] (5)
As shown above, Tinf contains three prompt tasks: event-type compatibility, argument com- patibility, and coreference judgment. As such, PTM can capture the associations among the pre- dictions of these three prompt tasks to improve ECR. Benefiting from the CoT-style template form, PTM can perform deeper and more thinking than direct prediction. Considering that "compatibil- ity" be expressed by numerous words in the vocabulary, we again use the semantical verbal- izer, i.e., Eq. (2), to create virtual label words for "compatible" and "incompatible" labels. Here, we use the manually constructed word set Vn. = Veomp U Vincom to initialize the label word em- beddings: Veomp = {same, related, relevant, similar, matching, matched]. Vincom (different, unrelated, irrelevant, dissimilar, mismatched). where Voomp and Vincom correspond to the labels "compatible" and "incompatible", respectively.
M. = M(e;, e5) mmype = M(h; h
Tupe Tupe
where h upe h2 Tupe are the mask token embed- dings used for predicting the event types of ev, and Ciy in the anchor template Tano- After obtaining the trigger and event-type matching features mmm and mp'pp: we utilize them to update all the three mask token embeddings in the inference template Tinf. helping the PTM finish the corresponding prompt tasks better:
Tupe T [H = (h Type
[MASK] MT upe W
(7)
Arg
(a Arg
[M[MSK]
[HASK]] Sem W.
(8)
-Coref
hT = [h [ m Sem W.
(9)
where [MASK]: [=
Coref are mask token em- beddings corresponding to the event-type compati- bility, argument compatibility, and coreference dis- crimination tasks, respectively, and W.. W., W. parameter matrices responsible for transform- ing tensor dimensions to that of PTT. Finally, we feed these updated mask token embeddings into the MLL head of the PTM for prediction. Before train- ing, we construct event-type compatibility labels according to whether em; and eny have the same
Type
3.4 Mask Token Embedding Updating
Previous fine-tuning work shows that tensor match- ing can effectively capture the semantic interac- tions of events (Xu et al., 2022), which is also help- ful for our three prompt tasks in Tin.. Therefore, we introduce similar semantic matching operations to update the mask word embeddings by inject- ing interactive features. In this way, the PTM can

---

event type, while argument compatibility and coref- erence prediction tasks use the coreference labels directly. To update the parameters, we calculate the cross-entropy losses between the predictions and the labels at these mask positions. Conveniently, we denote the coreference loss as Le: and combine the event-type compatibility and argument compat- ibility losses L,, La the compatibility loss fm.
computational cost, we apply undersampling on the training set, and only about 10% of the training data are finally used (details Appendix A). For event extraction, we directly use the triggers provided by Xu et al. (2022), and then apply the OmniEvent toolkit to extract the corresponding arguments and select the argument roles related to participants and locations. After obtaining the coreference predic- tions of all event mention pairs, we create the final event clusters using greedy clustering algorithm (Xu et al., 2022). Finally, we report the ECR per- formance using the official Reference Coreference Scorer', which employs four coreference metrics, including MUC (Vilain et al., 1995), B (Bagga and Baldwin, 1998), CEAF& (Luo, 2005), BLANC (Recasens and Hovy, 2011), and the unweighted average of their Fl scores (AVG).
3.5 Training
To simultaneously update parameters in the PTM (including MLM head) and the matching module, we jointly perform three tasks of event-type pre- diction, compatibility prediction, and coreference judgment (corresponding losses are Cs, Cm Le, re- spectively), and define the overall loss function as:
L ==
log(1 + Li)
(10)
We choose RoBERTa with open pre-trained pa- rameters* as our encoder, which has 24 layers, 1024 hiddens, and 16 heads. All new Icarnable tokens we add to the vocabulary have 102--dimess em- beddings. For all samples, we truncate the context around the two event mentions to make the final input sequence length not exceed 512. For the ten- matching operations, following previous work (Xu et al., 2022), we set the matching dimension and perspective number 64 and 128, respectively, and set the tensor factorization parameter to The batch size and the number of training epochs are 4 and 10, and an Adam optimizer with a learning rate of le-5 is applied to update all parameters. The random seed used in all components is set to 42.
(c(s.m.c)
In this way, the optimizer can automatically regulate the balances among these three tasks by weights i E {s. (s.m,e m, c). In addition, inspired by the trigger-mask mechanism (Liu et al., 2020; Xu et al., 2022), we propose trigger- maak regu- larization to enhance the robustness of our model. Specifically, during training, we simultaneously in- put another template whose triggers are all masked and ask the PTM to perform the same three tasks. This forces the PTM to mine clues from the context instead of merely memorizing the mapping from trigger pairs to coreferences. When predicting, the target positions of the event-type prediction and the two auxiliary compatibility tasks are all kept as [MASK], and we only need output the probability distribution of the label words in the mask position corresponding to the ECR task.
4.2 Experimental Results
We compare our proposed CorefPrompt with the following strong baselines under the same eval- uation settings: (i) the joint model Lu&Ng2021 (Lu and Ng, 2021b), which jointly models six re- lated event and entity tasks, and (ii) the pairwise model Xu2022 (Xu et al., 2022), which introduces Socument-level event encoding and event topic model. In addition, we build two pairwise base- lines, BERT and RoBERTa, that utilize the popu- lar BERT/ROBERTa model as the encoder. Specifi- cally, they first feed a segment that does not exceed the maximum length of the encoder, including two event mentions, into the BERT/RoBERTa model and then obtain the two event trigger representa-
4 Experimentation
4.1 Experimental Settings
Following previous work (Lu and Ng, 2021c; Xu et al., 2022), we choose KBP 2015 and KBP 2016 (Mitamura et al., 2015, 2016) as the training set (LDC2015E22 E68, E73, E94, and LDC2016E64) and use KBP 2017 (Mitamura et al., 2017) as the test set. The training set includes 817 documents annotated with 22894 event mentions distributed in 14794 clusters, and the test consists of 167 documents annotated with 4375 event mentions distributed in 2963 clusters. Following Lu and Ng (2021c), we select the same 82 documents from the training set for parameter tuning. To reduce the
h com/ Setm/TTT--KG/Ommiveeee hhttp:://gittub.com// refereece-corfferene--cree "https://huggingfacce http://ugggngfaaaaeccoorrretaaaa cabb

---

Model
MUC
B3
CEA.
BLA.
AVG
5.1
Prompt Tuning or Fine-tuning?
BERT
35.8
54.4
55.6
36.0
45.5
To demonstrate the effectiveness of prompt-based methods on the ECR task, we compare the follow- ing baselines, using BERT(B) and RoBERTa(Ro) as encoders: (i) Pairwise(B) and Pairwise(Ro):: fine- tuning pairwise models, which first utilize the en- coder and attention mechanism obtain event rep- resentations and then send event pair features to a scorer; (ii) Prompt(B) and Prompt(Ro): com- mon prompt methods, which convert ECR into a cloze-style MLM task by adding a template TECr before the original input; (iii) CorefPrompi( and CorefPrompt(Ro)): using the prompt specially de- signed for ECR to stimulate the potential of PTMs in judging coreference. The results in Table 2 show that benefiting from eliminating the gap between ECR and pre-training tasks, prompt-based methods can leverage PTM knowledge to make judgments. Therefore, Prompt(B) and Prompt(Ro) achieve 1.6 and 1.1 improvements on AVG-F1 over their fine- tuning counterparts, respectively, without perform- ing any tensor matching. However, these methods do not consider the characteristics of the ECR task and rely only on PTMs' understanding to make predictions. In contrast, our prompt adequately incorporates the critical points of ECR, e.g., argu- ment compatibility, to provide better guidance, thus achieving the best performance.
RoBERTa
37,9
55.9
57.3
38.3
47.3
Ng2N22021
45.2
54.7
53.8
38.2
48.0
Xu2022
46.2
57.4
59.0
42.0
51.2
CorefPrompt
45.3
57.5
59.9
42.3
51.3
Table I: Performance of all baselines and our Coref- Prompt on the KBP 2017 dataset.
tions ei, ej by an attention mechanism. Finally, the combined feature vector [e,; eyie, o ej] sent to an MLP to identify coreference.
Table I reports the performance of the four baselines and our CorefPrompt on the KBP 2017 dataset, and the results show that our proposed model achieves comparable performance to SOTA Xu2022, without using full-text level encoding, which demonstrates the effectiveness of our method in resolving event coreference. A detailed compari- son of the computational efficiency of our method and Xu2022 can be seen in Appendix B.
Benefiting from joint modeling multiple event tasks and introducing the powerful SpanBERT en- coder, Lu&Ng2021 performs better than BERT and RoBERTa, with a significant improvement of more than 7.3 on the MUC metric. However, all these methods model event mentions based only segment-level context; therefore, after Xu2022 introduces document-level event encoding based on full-text context using the Longformer encoder, the ECR performance improves consider- ably. Nevertheless, all of these baselines employ traditional fine-tuning frameworks that have large gap with the pre-training tasks of PTMs, neither completely leveraging the knowledge contained in PTMs nor directly incorporating human knowledge into model predictions. In contrast, our approach designs a specific prompt to motivate the poten- tial of PTMs in the ECR task, incorporating rich manually summarized rules to guide the model; therefore, it achieves comparable performance to Xu2022, based only on segment-level coding.
Model
MUC
B3
CEA.
BLA.
AVG
Pairwise(B)
40.2
53.7
55.0
35.1
46.0
Pairwise(Ro)
43.8
55.2
57.0
38.5
48.6
Prompt(B)
41.4
54.9
56.2
37.8
47.6
Prompt(Ro)
45.3
56.1
57.4
40.1
49.7
CorefPrompt(
41.5
55.4
56.9
38.9
48.2
Core(Prompt(Ro)
45.3
57.5
59.9
42.3
51.3
Table 2: Comparison of fine-tuning models and prompt- based methods.
To deeply compare these methods and exclude the effect of clustering, also report the event-
pair classification Fl-score in Table 3: (1) ALL: results of all event mention pairs; (2) results
event mentions with different argument states: (i) NoA: neither event has an argument; (ii) OneA: only one event contains arguments; (iii) BothA: both events contain arguments.
5 Analysis and Discussion
We first analyze the differences between prompt- based and fine-tuning methods in processing the ECR task, showing the advantages of prompting with well-designed templates, and then discuss the contribution of each component in our approach through ablation experiments.
Table 3 shows that the prompt-based methods still outperform the corresponding fine-tuning mod- els. In particular, the prompt-based methods allure out PTM's understanding of ECR, thus helping to judge events that contain rich argument informa-

---

Model
ALL
NoA
OneA
BothA
cates that it is difficult for PTM learn a suitable template for the complex ECR task automatically, and the template design is more dependent on man- ual constraints. Our approach incorporates much human knowledge, e.g., considering both event- type and argument compatibilities, thus improving judgments in both NoA and HasA cases.
(B))
32,6
34.1
33.2
32.1
Pairwise(Ro)
36.4
36.3
38.2
36.0
Prompt(B)
34.0
34.0
35.4
34.2
Prompt(Ro)
38.6
40.2
40.9
38.5
CorefPrompt(B)
34.6
35.7
35.7
34.7
Core(Prompt(Ro)
40.5
41.2
42.4
39.5
Table 3: Event-pair classification results of event tions with different argument states.
5.2 Ablation Study
To analyze the contribution of each component in our method, we design the following ablations: (i) -Pre: removing prefix template; (ii) -Anc: remov- ing anchor templates, obtaining event embeddings based on the triggers in the document; (iii) -Aux:
tion, Prompt(B) and Prompt(Ro) achieve im- provement of more than 2.0 in both OneA and BothA cases. This verifies that argument compati- bility is a crucial point for ECR. CorefPrompt not only guides PTM to focus on the argument infor- mation but also considers event-type compatibility, thus improving the judgment in all cases compared to common prompts. This validates the effective- ness of our prompt design in capturing event-type and argument compatibilities to improve ECR.
removing type compatibility and argument compat- ibility auxiliary tasks; (iv) -Reg: removing trigger- mask regularization; (v): -TM: removing tensor matching operations, predicting based on original [MASK] tokens. The results are shown in Table 5.
Model
MUC
B3
CEA.
BLA.
AVG
ALL
Full
45.3
57.5
59.9
42.3
51.3
40.5
Table 2 shows that the common prompts also
Pre
44.7
57.0
59.4
41.3
50.6
40.0
achieve acceptable performance, so we construct the following templates for comparison: (i) Con-
-Ac
44.5
56.9
58.8
41.0
50.3
38.4
Aux
43.9
56.7
59.1
40.8
50.1
38.3
nect: placing [MASK] between event mentions, and using descriptions "refer to" and "not refer to" to create virtual label words; (ii) Question: question- style prompt, which asks PTM whether two events are coreferential; (iii) Soft: wrapping event men- tions with learnable virtual tokens build the tem- plate. More details can be seen in Appendix C. The results are shown Table 4, where HasA indicates that at least one event contains arguments:
Reg
46.1
56.6
58.9
40.6
50.5
40.3
-TM
44.7
57.1
59.3
41.8
50.7
39.8
Table 5: Results of various model variants.
Table 5 shows that removing any part will hurt the ECR performance, especially -Anc and -Aux, causing large drops in event-pair classification F1- scores of 2.1 and 2.2, respectively. This illustrates that anchor templates and auxiliary compatibility tasks have the most significant impact on our model. In contrast, -TM brings only tiny drop of 0.6 and 0.7 in AVG-F1 and event pair F1-score. This sug- gests that our well-designed prompt can implicitly capture the interactions between event mentions without deliberately using the matching module.
Model
AVG
ALL
NoA
HasAA
Normal
49.7
38.6
40.2
39.7
Connect
50.0
38.3
37.8
39.7
Question
50.2
39.7
40.0
40.4
Soft
49.7
38.1
38.7
37.5
Ours
51.3
40.5
41.2
40.9
Table 4: Results using different prompts.
Since the main contribution of the anchor tem- plate is to mark the argument information, we take the variant whose anchor template only includes event type as the Base model and apply two ways to add event arguments: (i) +Arg: adding all rec- ognized arguments, i.e., our method; (ii) +Arg(B): keeping events have balanced argument informa- tion, i.e., if one event does not contain a specific argument role, the other will also not add. The event-pair results (P F1) are shown in Table 6.
Table 4 shows that, different from the general opinion that the template form has a significant in- fluence (Gao et al., 2021), all common templates achieve similar performance. This may be because ECR is complex task that relies on inference, it is challenging construct high-quality template, and merely changing the template form can only bring limited impact. Soft shows that adding learn- able tokens does not improve ECR, and event-pair classification performance even drops. This indi-
The results of +Arg and +Arg(B) show that after explicitly marking the arguments, improving not

---

Model
NoA
OneA
BothA
Conclusion
Base
45.27 /8.9331.//8.0
/40.7
40.2/38.7 39.4
+Arg
41.5/40.8/41.2
47.1/3827424
39.3/39.6/ 39.5
In this paper, we design a specific prompt to trans- form ECR into a mask language prediction task. As such, event modeling and coreference judgment can be performed simultancously based on a shared context. In addition, we introduce two auxiliary mask tasks, event-type compatibility and argument compatibility, to explicitly show the reasoning pro- cess of ECR, so that PTM can synthesize multi- dimensional event information to make the final predictions. Experimental results on the KBP 2017 dataset show that our method performs compara- bly to previous SOTA, without introducing full-text level encoding. In future work, will continue study how incorporate more human knowledge into the prompt- based approach for ECR.
+Arg(B)
40.0/42.3/4.1
44.27
38.3/40.7 39.3
Table 6: Results after adding arguments.
only the judgment of events with arguments but also those without, whose performance is greatly enhanced by 3.2 and 3.1, respectively. This may be because events with rich argument information are inherently easy to judge, especially when ar- guments are explicitly marked. In this way, the model can focus on distinguishing those "difficult" samples without argument information. Therefore. the less argument information the sample has, the more the performance improves (NoA > OneA BothA). Although +Arg(B) allows the model to compare event mentions evenly, this reduces the difference between events, making the model tend to judge samples as coreferential, improving the recall while gravely hurting the precision.
Limitations
Despite the simplicity and effectiveness of our ap- proach, it still suffers from two obvious shortcom- ings. First, since our model adopts a pipeline frame- work, we need to pre-identify the triggers in the document before constructing the prompt, which inevitably leads to error propagation. Therefore, how to design prompt that can jointly identify triggers and judge event coreference is still solved problem. Second, our experiments construct samples for every event mention pair in the doe- ument, resulting in a large training set. Although we greatly reduce the training data size by under- sampling, it still costs more training time than the slice or full-text encodings employed in previous studies. Therefore, how to design templates with higher event coding efficiency is another focus of our future work.
To evaluate the contributions of the two auxiliary prompt tasks, we take the variant whose inference template only contains the coreference task as Base and then: (i) +TypeMatch: adding event-type com- patibility task; (ii) Matghhtch: adding argument compatibility task; (iii) Arggt&ArgMatch:: adding these two simultaneously, i.e., our method. Table 7 shows the event-pain classification results.
Model
ALL 09.2/37.5 38.3
Base
+TypeMatch
39.4 / 40.4 39.9
+ArgMatch
41.5/ 38.1 39.7
+Tyee&ArMMaich
42.1 / 39.0/ 40.5
A Acknowledgements
Table Results after introducing auxiliary tasks.
The authors would like thank the three anony- mous reviewers for their comments on this paper. This research was supported by the National Natu- ral Science Foundation of China (Nos. 61836007, 62276177, and 62376181), and Project Funded by the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD).
Table shows that introducing type compatibil- ity and argument compatibility helps the model make predictions; the F1 scores increased by 1.6 and 1.4, respectively. The +TypeMatch provides the model with additional event type information beyond the original text, significantly improving recall by 2.9. While +ArgMatch guides the PTM to focus on distinguishing argument clues, resulting in considerable improvement in precision by 2.3. Our approach introduces these two auxiliary tasks simultancously so that they complement each other to achieve the best performance.
References
Amit Bagga and Breck Baldwin. 1998. Algorithms for scoring coreference chains. In The first international conference on language resources and evaluation workshop on linguistics coreference, pages 563-566.

---

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in Neural Information Processing Systems 33 (NeurIPS 2020), 33:1877-1901.
Huig- Huang and Nanyun Peng. 2021. Document-level event extraction with efficient end-to-end learning of cross-event dependencies. In Proceedings of the Third Workshop on Narrative Understanding, pages 36-47.
Yin Jou Huang. Jing Lu, Sadao Kurohashi, and Vincent Ng. 2019. Improving event coreference resolution by learning argument compatibility from unlabeled data, In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computa- rional Linguistics: Human Language Technologies. pages 785-795.
Xiang Chen, Ningyu Zhang, Xin Xie, Shumin Deng, Yunzhi Yao, Chuanqi Tan, Fei Huang, Luo Si, and Huajun Chen. 2022. Knowprompt: Knowledge- aware prompt tuning with synergistic optimization for relation extraction. In Proceedings of the ACM Web Conference 2022, pages 2778-2788.
Zheng Chen and Heng Ji. 2009. Graph- Graph-based event coreference resolution. In Proceedings of the 2009
Kian Kenyon-Dean, Jackie Chi Kit Cheung, and Doina Precup. 2018. Resolving event coreference with supervised representation learning and clustering- oriented regularization. In Proceedings of the Sev- enth Joint Conference on Lexical and Computational Semantics, pages 1-10.
Workshop on Graph-baste based Methods for Natural Lan- guage Processing, ACL-IJCNLP 2009, pages 54-57.
Zheng Chen, Heng Ji, and Robert M Haralick. 2009. A pairwise event coreference model, feature impact and evaluation for event coreference resolution. In Proceedings of the Workshop on Events in Emerging Text Types, pages 17-22.
Sebastian Krause, Feiyu Xu, Hans Uszkoreit, and Dirk Weissenborn. 2016. Event linking with sentential features from convolutional neural networks. In Pro- ceedings of The 20th SIGNLL Conference on Compu- tational Natural Language Learning, pages 239-249.
Yew Ken Chia, Lidong Bing, Soujanya Poria, and Luo Si. 2022. Relationprompt: Leveraging prompts to generate synthetic data for zero-shot relation triplet extraction. In Findings of the Association for Com- putational Linguistics: ACL. 2022, pages 45-57.
Samuel Kriman and Heng Ji. 2021. Joint detection and coreference resolution of entities and events with document-level context aggregation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natural Language Processing: Student Research Workshop, pages 174-179.
Leyang Cui, Yu Wu, Jian Liu, Sen Yang, and Yue Zhang. 2021. Template-basede named entity recognition us- ing bart. In Findings of the Association for Com- putational Linguistics: ACL-IJCNLP 2021, pages 1835-1845.
Agata Cybulska and Pick Vossen. 2015. Translating gramularity of event slots into features for event coref- erence resolution. In Proceedings of the the 3rd Work- shop on EVENTS: Definition, Detection, Coreference, and Representation, pages 1-10.
Tuan Lai, Heng Ji, Trung Bui, Quan Hung Tran, Franck Dernoncourt, and Walter Chang. 2021. A context- dependent gated module for incorporating symbolic semantics into event coreference resolution. In Pro- ceedings of the 2021 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3491-3499.
Lu Dai, Bang Wang, Wei Xiang, and Yijun Mo. 2022. Bi-directional iterative prompt-tuning for event argu- ment extraction. arXiv preprint arXiv:2210. 15844333.
Tianyu Gao, Adam Fisch, and Danqi Chen. 2021. Making pre-trained language models better few-shot learners. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the 11th International Joint Conference on Natu- ral Language Processing (Volume I: Long Papers), pages 3816 3816-3830.
I-Ta Lee, Maria Leonor Pacheco, and Dan Goldwasser. 2020. supervsd modeling of contextual- ized event embedding for discourse relations. In Findings of the Association for Computational Lin- guistics: EMNLP 2020, pages 1962-4972
Haochen Li, Tong Mo. Hongcheng Fan, Jingkun Wang. Jiaxi Wang, Fuhao Zhang, and Weiping Li. 2022. Kipt: Knowledge-injetted prompt tuning for event detection. In Proceedings of the 29th International Conference on Computational Linguistics, pages 1943-1952.
Xu Han, Weilin Zhao, Ning Ding, Zhiyuan Liu, and Maosong Sun. 2022. Ptr: Prompt tuning with rules for text classification. Al Open, 3:9.
I-Hung Hsu, Kuan-Hao Huang, Elizabeth Boschee, Scott Miller, Prem Natarajam, Kai- Wei Chang, and Nanyun Peng. 2022. Degree: A data-efficient generation based event extraction model. In Pro- ceedings of the 2022 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pages
Manling Li, Tengfei Ma, Mo Yu, Lingfei Wu, Tian Gao, Heng Ji, and Kathleen McKeown. 2021. Timeline summarization based on event graph compression via time-aware optimal transport. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6443-6456.
1890-1908.

---

Jian Liu, Yubo Chen, Kang Liu, Yantao Jia, and Zhicheng Sheng. 2020. How does context matter? on the robustness of event detection with context- selective mask generalization. In Findings of the Association for Computational Linguisties: EMNLP 2020, pages 2523-2533
Thien Huu Nguyen, Adam Meyers, and Ralph Grish- man. 2016. New york university 2016 system for kbp event nugget: A deep learning approach. In Proceedings of the Text Analysis Conference.
Shon Otmazgin, Arie Cattan, and Yoav Goldherg. 2022. Lingmess: Linguistically informed multi expert scorers for coreference resolution. arXiv preprint arXiv:2205. 12644
Zhengzhong Liu, Jun Araki, Eduard H Hovy, and Teruko Mitamura. 2014. Supervised within- document event coreference using information prop- agation. In Proceedings of the 9th International Conference on Language Resources and Evaluation, pages 4533-4544,
Haoruo Peng, Yangqiu Song. and Dan Roth. 2016. Event detection and co-reference with minimal su- pervision. In Proceedings of the 2016 conference on empirical methods in natural language processing, pages 392 402
Jing Lu and Vincent Ng. 2021a. Span-based event coreference resolution. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 13489-
Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, and Huajun Chen. 2022. Reasoning with lan- guage model prompting: A survey. arXiv preprint arXiv:2212.09M0
13497.
Jing Lu and Vincent Ng. 202lb. Constrained multi- task learning for event coreference resolution. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 1500-4514,
Marta Recasens and Eduard Hovy. 2011. Blanc: Im- plementing the rand index for coreference evaluation. Natural language engineering, 17(4):485-510.
Shirong Shen, Heng Zhou, Tongtong Wu, and Guilin Qi. 2022. Event causality identification via derivative prompt joint learning. In Proceedings of the 29th International Conference on Computational Linguis- tics, pages 2288-2299.
Jing Lu and Vincent Ng. 2021c. Conundrums in event coreference resolution: Making sense of the state of the art. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1368- 1380..
Yifan Sun, Changmao Cheng. Yuhan Zhang. Chi Zhang, Liang Zheng, Zhongdao Wang. and Yichen Wei. 2020. Circle loss: A unified perspective of pair simi- larity optimization. In Proceedings of the IEFF/CVF conference on computer vision and pattern recogni- rion, pages 6398-6407.
Yaojie Lu, Qing Liu, Dai Dai, Xinyan Xiao, Hongyu Lin, Xianpei Han, Le Sun, and Hua Wu. 2022. Uni- fied structure generation for universal information extraction. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics, pages 5755-5772.
Hieu Minh Tran, Duy Phung, and Thien Huu Nguyen. 2021. Exploiting document structures and cluster consistencies for event coreference resolution. In Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguistics and the I 11th International Joint Conference on Natural Language Processing, pages 4840-4850.
Xiaoqiang Luo. 2005. On coreference resolution perfor- mance metrics. In Proceedings of Human Language Technology Conference and Conference on Empiri- cal Methods in Natural Language Processing, pages 25-32.
Yubo Ma, Zehao Wang, Yixin Cao, Mukai Li, Meiqi
Chen, Kun Wang, and Jing Shao. 2022. Prompt for extraction? paie: Prompting argument interaction for event argument extraction. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, pages 6759-6774.
Mare Vilain, John D Burger, John Aberdeen, Dennis Connolly, and Lynette Hirschman. 1995. A model- theoretic coreference scoring scheme. In Proceed- ings of the 6th Message Understanding Conference.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoe Le, and Denny Zhou. 2022. of-in-of-thoughh prompting elicits reasoning in large language models. arXiv preprint a 11903.
Teruko Mitamura, Zhengzhong Liu, and Eduard H Hovy. 2015. Overview of TAC KBP 2015 event nugget track. In Proceedings of the Text Analysis Confer- ence.
Dennis L. Wilson. 1972. Asymptotic properties of near- est neighbor rules using edited data. IEEF Trans- actions on Systems, Man, and Cybernetics, (3):488- 421.
Teruko Mitamura, Zhengzhong Liu, and Eduard H Hovy. 2016. Overview of tac-kbp 2016 event nugget track. In Proceedings of the Text Analysis Conference.
Teruko Mitamura, Zhengzhong Liu, and Eduard H Hovy. 2017. Overview of tac kbp 2015 event nugget track. In Events Detection, Coreference and Sequencing: What's 's Next? Overview of the TAC KBP 2017 Event Track.
Sheng Xu, Peifeng Li, and Qiaoming Zhu. 2022. Im- proving event coreference resolution using document- level and topic-level information. In Proceedings of the 2022 Conference on Empirical Methods in Natu- ral Language Processing, pages 6765-6775

---

Bishan Yang. Claire Cardie, and Peter Frazier. 2015. hierarchical distance- dettnne-dependent bayesian model for event coreference resolution. Transactions of the Association for Computational Linguistics, 3:517- 528.
Le log
1+
Σ (ean
Sicheng Yang and Dandan Song. 2022. Fpe: Fine- tuning with prompt curriculum for relation extrac- tion. In Proceedings of the 2nd Conference of the Asia-Pucific Chapter of the Association for Compu- tational Linguistics and the 12th International Joint Conference on Natural Language Processing, pages 1065-1077.
(11)
where Spos and Gney represent coreferential and non-coreferential event sets, respectively, and Cs Cjj, Ck: ej are event representations.
The statisties of the training data sampled by CorefENN-1, CorefENN-2, and CorefNM are shown in Table 8, where "No" represents the origi- nal training set.
Yutao Zeng, Xiaolong Jin, Saiping Guan, Jiafeng Guo, and Xueqi Cheng. 2020. Event coreference resolu- tion with their paraphrases and argument-aware em- heddings. In Proceedings of the 28th International Conference on Computational Linguistics, pages 3084-3094.
Sampling
Param
Coref
Non-Coref
All
No
23,138
415,042
188,180
CorefENN-1
k=2
23.138
16,522
39,660
k=3
23,138
31,189
54,327
Jianping Zhang and Inderjeet Mani. 2003. knn approach to unbalanced data distributions: A case study involv- ing information extraction. In Proceedings of the Twentieth International Conference International Conference on Machine Learning O33), Work- shop Learning from Imbalanced Data Sets.
k=4
23,138
48,726
71,864
CoretENN-2
7=0.25
23,138
13,439
36,577
7=0.2
23,138
23,697
46,835
7=0.15
23.138
42,471
65,609
CorefNM
k=2
23,138
19,300
42,438
k=3
23,138
28,716
51,854
k=4
23,138
38,175
61,313
A Undersampling for ECR
Table 8:
Statistics of the training set constructed by different sampling strategies.
To reduce the training cost, design three dersampling strategies for the ECR task: (1) CorefENN-I and CorefENN-2: dropping samples that are easy to judge. inspired by ENN (Wil- son, 1972), where (i) CorefENN-1: If the top k events with the highest similarity to an event have the same coreference, dropping this event; (ii) Cor filtering out the negative sam- ples whose event pair similarity is lower than the threshold 7: (2) CorefNM: selecting representative negative samples, inspired by NearMiss (Zhang and Mani, 2003). Specifically, for each event, only the top & non-coreferantial events with the largest similarity are selected to pair with this event.
To balance the positive and negative samples and make the datasets obtained by different sampling have similar sizes, we finally set in CorefENN- 1 to 3, in CorefENN-2 to be 0.2, and set k in CorefNM to be 3. Under these settings, the dis- tribution of events on the long chain, short chain, and singleton is shown in Table 9, where (i) Single- ton: at least one event is singleton; (ii) Long: least one event comes from long chain (length 10), and no event the event pair is a singleton; (iii) Short: other samples. Here, "Random" repre- sents random undersampling, directly equalizing the number of positive and negative samples.
Considering that all these strategies exploit the similarity of events, we first build an event en- coder based on the Longformer model and atten- tion mechanism, producing similar embeddings for coreferential event mentions. As non-coreferential event mentions may also have similar contexts, we choose Circle Loss (Sun al., 2020) to op- timize the model such that coreferential events would have higher embedding similarities than non- coreferential ones:
Sampling Random
Singleton
Long
Short
57.6%
21.4%
21.0%
CorefENN-1
0%
0.5%
99.5%
CorefENN-2
62.5%
12.2%
25.3%
CorefNM
71.7%
5.4%
22.9%
Table 9:
Event distribution under different sampling.
The filtering of CoreiENN-1 drops events from long chains and singletons, resulting in nearly all

---

Prompt
Template
Normal
the following text, events expressed by [E1S] ev; [E1E] and [E2S] evi [E2E]
refer to [MASK] event: (Segment)
Connect
In the following text, the event expressed by [E1S] evi [E1E] [MASK] the event.
expressed by [E2S] ev; (EEE]:(Seggmeet
Question
In the following text, do events expressed by [E1S] [E1E] and [E2S] ev [E2E]
refer to the same (aaaaaMMAAASKSSammmaaa the following text, [L1][E1S] ev; [E1E][L2]][L3]IE2S] evj [E2E][L4]
Soft
[L5] [ASK][[6]:[Seegeent
Table 11: Different common templates.
selected events coming from short chains. Both CoreIENN-2 and CoreINM focus on sampling non- coreferential event pairs with high similarity; there- fore, the proportion of singletons is higher than that of random sampling. The results of using these different undersamplings are shown in Table 10:
22 GB, enabling it to run on a standard graphics card (such as the RTX3090).
As our method generates samples for every event mention pair, it requires more training time com- pared with Xu2022. The time cost for Xu2022 trained on A100-80G is approximately 6.7 hours (4 minutes/epoch, 100 epochs, 400 minutes total), while that of our method on the RTX3090 about
Model
MUC
B3
CEA.
BLA.
AVG
Random
44.4
54.1
54.7
39.4
48.1
18 hours (1 (110 minutes/epoch, 10 epochs, 1 100 min-
CorefENN-1
40.0
46.8
41.8
33.1
40.4
utes total). However, if we also seleet the A100- 80G for training, the time could be reduced to ap- proximately 9 hours (54 minutes/epoch, 10 epochs, 540 minutes total), similar to that of Xu2022.
CorefENN-2
44.8
56.4
58.6
40.3
50.0
CorefNM
45.3
57.5
59.9
42.3
51.3
Table 10: Results with different undersampling.
C Common Prompts
Table 10 shows that, compared to randomly se- lecting negative samples, the CorefNM strategy we selects representative samples based on event similarities, thus achieving performance improvement of 3.2. CorefENN-1 performs the worst due to missing a large number of events from long chains and singletons, which account for a high proportion of the dataset. Like CoreiNM, CoreiENN-2 retains non-coreferential event pairs with high similarity; hence, its performance is also higher than random sampling.
As shown in Table 11, we construct various com- mon prompts for the ECR task, including (i) Nor- mal: normal template, whose label words di- rectly from the vocabulary, "same" corresponds to coreferential, "different" corresponds to non- coreferential; (ii) Connect: placing the [MASK] ken between two event mentions, using the descrip- tions "refer to" and "not refer to" to initialize the embeddings of virtual label words; (iii) Question: asking the PTM whether the two event mentions are coreferential, and the PTM answers "yes" for coreferential and "no" for non-coreferential;: (iv) Soft: using learnable tokens that wrap two event mentions replace the manually written words, such, the PTM can automatically learn the appropri- ate template for ECR. The complete form of these templates is shown in Table 11, where Segment represents the original input, and [L1] ~ [L6] are learnable tokens.
B Computational Efficiency Comparison
Compared with the previous SOTA model Xu2022 (Xu al., 2022), our model's advantage sig- nificantly reducing the space complexity without compromising performance. The document-level event encoder used in Xu2022 (i.e., Forgerrmer) requires processing the entire document at once, re- sulting in high GPU memory usage (approximately 44 GB) and necessitating powerful graphics card (e.g., A 100-80G) for operation. This requirement poses an unfriendly challenge researchers or in- stitutions with limited computing resources. Our segment-level encoder requires only approximately half the memory of the Longformer encoder, about


Reviews:
[{"review_id": "2lQGxizOtb", "reviewer": "Reviewer_B9w6", "summary": "", "questions": "Question A: What are the compute requirements? Training and inference times?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your valuable comments.\n\n**Q1: There were no justifications given about the method's simplicity when compared to previous SOTA method. Why use only segment? What are the compute requirements? Training and inference times?**\n\n**A1**: Compared with the previous SOTA model by Xu et al. (2022), our model's advantage is significantly reducing space complexity without compromising performance. The document-level event encoder used in SOTA (i.e., LongFormer) requires processing the entire document at once, resulting in high GPU memory usage (approximately 44GB) and necessitating a powerful graphics card (e.g., A100-80G) for operation. This requirement poses an unfriendly challenge to researchers or institutions with limited computing resources. Our segment-level encoder requires only approximately half the memory of Longformer, approximately 22GB, enabling it to run on a standard graphics card (such as the RTX 3090).\n\nAs our method generates samples for every event mention pair, it requires more training time. The time cost for the previous SOTA model (Xu et al. 2022) trained on A100-80G is approximately 6.7 hours (4 minutes/epoch, 100 epochs, 400 minutes total), while that of our method on the RTX3090 is approximately 18 hours (110 minutes/epoch, 10 epochs, 1100 minutes total). However, if we select the A100-80G for training, the time could be reduced to approximately 9 hours (54 minutes/epoch, 10 epochs, 540 minutes total), similar to that of the SOTA.\n\nWe will include these comparisons in the revised version.\n\n**Q2: There is a limitation the paper puts itself in when tackling new event types. It seems like their methodology would be more far-reaching if the authors could address how they would tackle new event types in this paper.**\n\n**A2**: Yes. Event coreference resolution with open event types is more aligned with real-world scenarios, which is a focus of our future work. However, since event coreference resolution is challenging (even though the performance of recent state-of-the-art models is still far from practical), most current research is carried out in scenarios with fixed event types. Therefore, to compare with the state-of-the-art models, we adhere to the setting of fixed event types. \n\nIn addition, our model can handle unseen event types to some extent by converting multiple event tasks into MLM tasks. During prediction, all task positions except for coreference judgment, such as event type prediction, need only be kept as [MASK] (Line 403-408) in the model. Therefore, while our prompting method has only been trained on the KBP dataset with fixed event types, the pre-trained model (PTM) may autonomously predict these mask tokens as corresponding label words (other words in the vocabulary) when encountering unseen event types, drawing on its own knowledge.\n\n**Q3: Only English corpora experiments should be a limitation.**\n\n**A3**: Yes, evaluating our model across various languages can establish its generality. However, while the present prominent KBP datasets comprise non-English languages such as Chinese and Spanish, recent studies in this field have concentrated solely on English due to the small size of non-English data. For instance, the number of Chinese documents in both the KBP 2016 and 2017 datasets is merely 334, which only accounts for approximately one-third of the English data size, and is inadequate to support contemporary deep learning techniques.\n\n**Q4: The paper should be made clearer for audiences who are not familiar with this work. Suggestions include adding examples, additional details about the corpora, etc.**\n\n**A4:** Yes. We will add an example in the Introduction section to explain what event coreference resolution is as follows, where \"[]\" is used to tag the annotated spans and \"{}\" is used to tag annotated labels. \n\n[Former Pakistani dancing girl]{arg1} commits [suicide]{ev1} 12 years after horrific [acid]{arg2} [attack]{ev2} which [left]{ev3} [her]{arg3} looking 'not human'. [She]{arg4} had undergone 39 separate surgeries to repair [damage]{ev4}. Leapt to [her]{arg5} [death]{ev5} from [sixth floor Rome building]{arg6} [earlier this month]{arg7}. [Her ex-husband]{arg8} was [charged]{ev6} with [attempted murder]{arg9} in [2002]{arg10} but has since been [acquitted]{ev7}. \n\nThe example above contains seven event mentions (marked as ev1-7) and ten entities (marked arg1-11) that serve as arguments. Among them, the die event mention ev1 with the argument arg1 and the die event mention ev5 with the arguments arg5, arg6, and arg7 are coreferential, and both of them describe the girl's suicide by jumping off a building; the injury event mention ev3 with the arguments arg2 and arg3 and the injury event ev4 with the argument arg4 are coreferential, and both of them describe the girl's disfigurement; other events are singletons. \n\nWe will also show the specific template of this example in the Model section corresponding to Figure 1. \n\nWe inadvertently omitted important citations on the KBP corpus in Section 4.1. In the revised version, we will add these citations [Mitamura et al., 2015, 2016, 2017] and give a more detailed introduction to the KBP dataset.\n\n[Mitamura et al., 2015] Teruko Mitamura, Zhengzhong Liu, and Eduard H Hovy. 2015. Overview of TAC KBP 2015 event nugget track. In Proceedings of the Text Analysis Conference.  \n[Mitamura et al., 2016] Teruko Mitamura, Zhengzhong Liu, and Eduard H Hovy. 2016. Overview of TAC KBP 2016 event nugget track. In Proceedings of the Text Analysis Conference.  \n[Mitamura et al., 2017] Teruko Mitamura, Zhengzhong Liu, and Eduard H Hovy. 2017. Overview of TAC KBP 2015 event nugget track. In Events Detection, Coreference and Sequencing: What’s Next? Overview of the TAC KBP 2017 Event Track.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- nice novel design bringing together ideas from various different previous works, and a solid experimental setup\n- comprehensive ablation study\n- well-written and clear (for an audience familiar with this work)\n- serves as a basis for modeling other event features as cloze task", "reasons_to_reject": "- There were no proper justifications given about the method's simplicity when compared to previous SOTA method. Is it more computationally efficient than previous work? Why use only segment when the previous method got a big bump in performance with the entire document? Without the justifications, using only segments in this work seems wasteful.\n- While the joint modeling of event types with coreference is ingenious, there is a limitation the paper puts itself in when tackling new event types. It seems like their methodology would be more far-reaching (on more than 1 dataset) if the authors could address how they would tackle new event types in this paper. Since they don't do it, it seems like a missed opportunity.", "questions_to_authors": "Question A: What are the compute requirements? Training and inference times?", "paper_topic_and_main_contributions": "This paper presents a novel approach to tackling within-document Event Coreference Resolution (ECR). The authors achieve this by reimagining the coreference prediction task as a cloze-style Masked Language Model (MLM) task. Through this technique, they effectively merge ECR with event type extraction and the prediction of event type and argument compatibility within the MLM framework using carefully designed prompt templates. This integration seemingly facilitates their system to adeptly grasp the intricate rationale behind event types and arguments compatibilities when making coreference decisions. They finally achieve near SOTA results for a specific dataset with this methodology while only using the segments instead of the entire document.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "-  line 06: relies -> rely\n- line 026: Why not just call it Within-document Event Coreference Resolution? changing terminology might throw off the readers \n- Only English corpora experiments should be a limitation\n- The paper in general should be made clearer for audiences who are not familiar with this work. Suggestions include adding examples (textual) in Figure 1, additional details about the corpora used, an explanation of how training and inference algorithms are different, etc."}}, {"review_id": "NRHLvaVLTB", "reviewer": "Reviewer_PPJ5", "summary": "", "questions": "- Granted that you produce a novel methodology using prompt tuning, but how do you know this Large Language Model has not seen the testing data during its training over the entire internet-crawled data? I mean, this is a pertinent deeper issue with the current state of the art, but my point is you must at east mention that as a limitation.\n\n", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your valuable comments.\n\n**Q1: In abstract you don't need to mention specific datasets. Just say performs well in a state of the art benchmark. Line 121 your method performing well is not your contribution. That is the result.**\n\n**A1**: Yes, we will revise the last sentence of the abstract as follows:\n\nExperimental results show that our method CorefPrompt performs well in a state-of-the-art benchmark.\n\nWe will also delete “Our method is simple yet effective, achieving comparable performance to SOTA.” from our contributions.\n\n**Q2: Introduction section does not read like an introduction and more like a related work section. Does not motivate the problem well. You keep mentioning encoding first and then scoring, but does not explain anywhere what it means.**\n\n**A2**: Yes. We will revise the third and the fourth paragraphs to introduce two motivations more clearly as follows:\n\nFirst, we note that most previous work follows the \"encoding-first, then-scoring\" framework, wherein they first use encoders (e.g., BERT, SpanBERT, Longformer, etc.) to encode the text and obtain event mention embeddings, and then apply a scorer to evaluate the coreference score of event pairs based on these learned embeddings. This results in the issue of \"information blocking.\" Essentially, since the scorer solely utilizes the learned embeddings as inputs, almost the entire coreference determination relies on the event encoding. However, the event encoding is performed independently, without direct influence from coreference judgment. As a result, the encoder may not accurately capture contextual information that is crucial for identifying event coreference. Especially for coreferential event pairs with unbalanced information, the learned event embeddings may significantly differ if one event has rich arguments (participants) while the other has only a vague event trigger (verb). To address this issue, we convert ECR into a mask language prediction task, using a well-designed prompt. Thus, the event modeling and coreference judgment can conveniently interact based on a fully shared context to improve each other.\n\nSecond, in order to restrict the coreference judgment, most previous work has often required designing special model structures to guide the model to focus on the compatibility of event elements. This approach relies on a large amount of training, and cannot guarantee that the model can finally capture the compatibility features or understand the association between compatibilities and coreference. In the worst cases, the automatically captured features focus on other aspects and fail to discover the association between compatibilities and coreference due to the bias of interaction features or noise in data. In this paper, we introduce two auxiliary prompt tasks, event-type compatibility and argument compatibility, to explicitly demonstrate the inference process of coreference judgment in the template and guide the model to make final decisions based on these compatibilities. Taking advantage of the convenience of incorporating human knowledge into a prompt-based approach, our method can navigate the model's focus on event type and argument compatibilities using templates in natural language. This not only utilizes the discriminant rules outlined by annotators, but also adjusts to different levels of compatibility with the assistance of soft label words.\n\nWe will also reorganize the introduction part of previous work around these two issues.\n\n**Q3: Can you add one solid example of event coreference resolution in Introduction. Figure 1: is way too much information without absolutely any explanation whatsoever.**\n\n**A3**: Yes. We will add an example in the Introduction section to explain what event coreference resolution is as follows, where \"[]\" is used to tag the annotated spans and \"{}\" is used to tag annotated labels.\n\n[Former Pakistani dancing girl]{arg1} commits [suicide]{ev1} 12 years after horrific [acid]{arg2} [attack]{ev2} which [left]{ev3} [her]{arg3} looking 'not human'. [She]{arg4} had undergone 39 separate surgeries to repair [damage]{ev4}. Leapt to [her]{arg5} [death]{ev5} from [sixth floor Rome building]{arg6} [earlier this month]{arg7}. [Her ex-husband]{arg8} was [charged]{ev6} with [attempted murder]{arg9} in [2002]{arg10} but has since been [acquitted]{ev7}.\n\nThe example above contains seven event mentions (marked as ev1-7) and ten entities (marked arg1-11) that serve as arguments. Among them, the die event mention ev1 with the argument arg1 and the die event mention ev5 with the arguments arg5, arg6, and arg7 are coreferential, and both of them describe the girl's suicide by jumping off a building; the injury event mention ev3 with the arguments arg2 and arg3 and the injury event ev4 with the argument arg4 are coreferential, and both of them describe the girl's disfigurement; other events are singletons. \n\nWe will also show the specific template of this example in the Model section corresponding to Figure 1. \n\n**Q4: Line 413 you throw a ton of names without any reference or explanation.**\n\n**A4**: We inadvertently omitted important citations on the KBP corpus in Section 4.1. In the revised version, we will add these citations [Mitamura et al., 2015, 2016, 2017] and give a more detailed introduction to the KBP dataset.\n\n[Mitamura et al., 2015] Teruko Mitamura, Zhengzhong Liu, and Eduard H Hovy. 2015. Overview of TAC KBP 2015 event nugget track. In Proceedings of the Text Analysis Conference.  \n[Mitamura et al., 2016] Teruko Mitamura, Zhengzhong Liu, and Eduard H Hovy. 2016. Overview of TAC KBP 2016 event nugget track. In Proceedings of the Text Analysis Conference.  \n[Mitamura et al., 2017] Teruko Mitamura, Zhengzhong Liu, and Eduard H Hovy. 2017. Overview of TAC KBP 2015 event nugget track. In Events Detection, Coreference and Sequencing: What’s Next? Overview of the TAC KBP 2017 Event Track.\n\n**Q5: You suddenly jump into a bunch of math or minor details without explaining well what the model does in plain text/no jargon. Refer to the motivation issue mentioned above which seems to be a pertinent theme throughout the paper. Figure 1: is way too much information without absolutely any explanation whatsoever.**\n\n**A5**: Yes. Our introduction to the model needs better organization. Specifically, in lines 192-206, we merged the introduction to prompting with our method, which is only a component of our entire framework. In the revised version, we will create a subsection exclusively focusing on introducing prompting. Secondly, we have not comprehensively described our method regarding motivations. To address this, we will include the following paragraph, demonstrating how our model in Figure 1 can tackle the two motivations mentioned in Section 1.\n\nFor the input event mention pair (e_i, e_j), we construct three corresponding templates: the prefix template T_pre, the anchor template T_anc, and the inference template T_inf. These templates respectively add guidance, encode events (including predicting event types), and discriminate coreference (including judging type and argument compatibilities). Then, we embed these templates into the segment hosting the two event mentions, converting the sample into a prompt containing multiple [mask] tokens. Since all event tasks are completed simultaneously in the same template, multiple steps, such as event modeling and coreference judgment, can interact conveniently based on a fully shared context, reducing the \"information blocking\" problem that existed in previous work. Finally, we send the entire prompt to the PTM and use the PTM's MLM head to predict the results of all tasks. We guide the PTM's encoding with the prefix template and explicitly demonstrate the coreference judgment reasoning process with the inference template, fully incorporating human knowledge into the model's predictions.\n\nThe caption for Figure 1 is too brief, so we will add a simplified version of the complete process to the caption.\n\n**Q6: How do you know this Large Language Model has not seen the testing data during its training over the entire internet-crawled data? My point is you must at east mention that as a limitation.**\n\n**A6**: Yes, in theory, all pre-trained models carry the risk of information leakage, which we will address in the limitations section. However, our model solely employs RoBERTa, one of the traditional PTMs. Therefore, this issue is not significant, given that traditional PTMs' (such as BERT and RoBERTa) pre-training tasks don't involve entity or event coreference resolution. While the pre-training process for popular LLMs such as GPT3 and LLaMa may involve entity coreference or event extraction, their risk of information leakage will significantly increase.\n\n**Q7: Line 475- I don't understand why you say comparable performance. Your work actually beats the SOTA with a good margin.**\n\n**A7**: You are right. Our model CorefPrompt outperforms the SOTA by 0.1 on AVG.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper introduces a novel methodology for Event Coreference Resolution and proves empirically that the solution is better than SOTA. The solution idea is novel, uses state of the art, and is a thoroughly written comprehensive paper from an engineering perspective considering all possible evaluation methodologies.", "reasons_to_reject": "- Core solution idea is probably good however the paper is not at all well written/motivated/it is not a smooth reading experience. The paper should not be accepted in this form/ needs a thorough revamping in the first 2 pages, despite this being a thorough engineering work\n\nAbstract\n- Line 022- what is KBP 2017. You are assuming every reader knows what it is. Further, in abstract you don't need to mention specific datasets. Just say performs well in a state of the art benchmark.\n\nIntroduction \n\n- Introduction section does not read like an introduction and more like a related work section. Does not motivate the problem well\n\n- Can you add one solid example of event coreference resolution in Introduction. Maybe at line 27 example. 1) Not everyone who is going to read this paper is going to know what it is 2) a research paper needs to be self contained as much as possible.\n\nE.g. Like the one in the introduction section of this work: Lu, Jing, and Vincent Ng. \"Span-based event coreference resolution.\" Proceedings of the AAAI conference on artificial intelligence. Vol. 35. No. 15. 2021.\n\n- You keep mentioning encoding first and then scoring, but does not explain anywhere what it means. Refer comment above about paper being self contained\n\n- Line 121- again, your method performing well is not your contribution. That is the result. \n\n- Line 413- you throw a ton of names without any reference or explanation \n\n- Line 475- i don't understand why you say comparable performance. Your work actually beats the SOTA with a good margin. \n\nModel:\n\nYou suddenly jump into a bunch of math/minor details without explaining well what the model does in plain text/no jargon. Refer to the motivation issue mentioned above which seems to be a pertinent theme throughout the paper. This currently reads as a pure engineering paper and not an NLP paper. \n\n", "questions_to_authors": "- Granted that you produce a novel methodology using prompt tuning, but how do you know this Large Language Model has not seen the testing data during its training over the entire internet-crawled data? I mean, this is a pertinent deeper issue with the current state of the art, but my point is you must at east mention that as a limitation.\n\n", "paper_topic_and_main_contributions": "This paper is about a novel solution to the event coreference resolution problem. The paper proposes a prompt-based approach Coref-Prompt to transform ECR into a cloze-style MLM (Masked Language Model) task. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "\n-Figure 1: is way too much information without absolutely any explanation whatsoever. A figure and its caption together must be completely self contained. \n\n- Line 111- what is KBP 2017. Either add a citation. Or don't mention the results in Intro. You don't need to talk about the effectiveness of your work in intro- where you just have to introduce the problem thoroughly, which you don't. \n\n"}}, {"review_id": "vu7vPZWb1u", "reviewer": "Reviewer_TxVr", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your valuable comments.\n\n**Q1: The motivation of employing the prompt-based method is not clear especially in the introduction section. One might be curious why the prompt-based method is suitable for the ECR task from a technical perspective?**\n\n**A1**: Yes. We will revise the third and the fourth paragraphs to introduce two motivations more clearly as follows:\n\nFirst, we note that most previous work follows the \"encoding-first, then-scoring\" framework, wherein they first use encoders (e.g., BERT, SpanBERT, Longformer, etc.) to encode the text and obtain event mention embeddings, and then apply a scorer to evaluate the coreference score of event pairs based on these learned embeddings. This results in the issue of \"information blocking.\" Essentially, since the scorer solely utilizes the learned embeddings as inputs, almost the entire coreference determination relies on the event encoding. However, the event encoding is performed independently, without direct influence from coreference judgment. As a result, the encoder may not accurately capture contextual information that is crucial for identifying event coreference. Especially for coreferential event pairs with unbalanced information, the learned event embeddings may significantly differ if one event has rich arguments (participants) while the other has only a vague event trigger (verb). To address this issue, we convert ECR into a mask language prediction task, using a well-designed prompt. Thus, the event modeling and coreference judgment can conveniently interact based on a fully shared context to improve each other.\n\nSecond, in order to restrict the coreference judgment, most previous work has often required designing special model structures to guide the model to focus on the compatibility of event elements. This approach relies on a large amount of training, and cannot guarantee that the model can finally capture the compatibility features or understand the association between compatibilities and coreference. In the worst cases, the automatically captured features focus on other aspects and fail to discover the association between compatibilities and coreference due to the bias of interaction features or noise in data. In this paper, we introduce two auxiliary prompt tasks, event-type compatibility and argument compatibility, to explicitly demonstrate the inference process of coreference judgment in the template and guide the model to make final decisions based on these compatibilities. Taking advantage of the convenience of incorporating human knowledge into a prompt-based approach, our method can navigate the model's focus on event type and argument compatibilities using templates in natural language. This not only utilizes the discriminant rules outlined by annotators, but also adjusts to different levels of compatibility with the assistance of soft label words.\n\n**Q2: The method only has 0.1 F1 score improvement compared with the SOTA. It is not sufficient to show the effectiveness of this method. Meanwhile, the proposed method costs more training time than the previous methods.**\n\n**A2**: Compared with the previous SOTA model by Xu et al. (2022), our model's advantage is significantly reducing space complexity without compromising performance. The document-level event encoder used in SOTA (i.e., LongFormer) requires processing the entire document at once, resulting in high GPU memory usage (approximately 44GB) and necessitating a powerful graphics card (e.g., A100-80G) for operation. This requirement poses an unfriendly challenge to researchers or institutions with limited computing resources. Our segment-level encoder requires only approximately half the memory of Longformer, approximately 22GB, enabling it to run on a standard graphics card (such as the RTX 3090).\n\nAs our method generates samples for every event mention pair, it requires more training time. The time cost for the previous SOTA model (Xu et al. 2022) trained on A100-80G is approximately 6.7 hours (4 minutes/epoch, 100 epochs, 400 minutes total), while that of our method on the RTX3090 is approximately 18 hours (110 minutes/epoch, 10 epochs, 1100 minutes total). However, if we select the A100-80G for training, the time could be reduced to approximately 9 hours (54 minutes/epoch, 10 epochs, 540 minutes total), similar to that of the SOTA.\n\nWe will include these comparisons in the revised version.\n\nMoreover, compared to the previous SOTA (Lu and Ng, 2021) that also employs segment-level context, our approach shows a significant improvement, with an average F1 score increase of 3.3. Our approach can be seen as a replacement for document-level event encoding to some extent, as we compensate for the limitations of segment-level event encoding by integrating human guidance and leveraging the knowledge embedded in PTM. This permits our model to retain exceptional performance while notably decreasing GPU memory usage.\n\n**Q3: I don't agree the claim presented in the lines 38-40 about entity coreference resolution. Semantic matching is only one of aspects need to be considered in the coreference resolution task. In addition, coreference resolution system also needs to consider compatibilities, such as agreement of number, gender and entity type.**\n\n**A3**: Yes. Ambiguity arises here due to inaccuracies in our descriptions. In fact, we want to express that event coreference resolution is more challenging than entity coreference resolution due to the complex event structures, i.e., triggers and corresponding arguments are loosely distributed in the text. We will fix this problem in the revised paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The analysis and discussion section presents more valuable information.", "reasons_to_reject": "The motivation of employing the prompt-based method is not clear especially in the introduction section. One might be curious why the prompt-based method is suitable for the ECR task from a technical perspective? It is not necessary to challenge previous studies on points that are proposed in this work unless there is a big improvement in performance causing by those points. But unfortunately, the method only has 0.1 F1 score improvement compared with the SOTA. It is not sufficient to show the effectiveness of this method. Meanwhile, the proposed method costs more training time than the previous methods. In short, I didn't observe many advantages of this work from the modeling perspective. However, the paper looks more like investigating the differences between prompt-based and fine-tuning methods on the ECR task. It will be more interesting to analyse in what cases the methods work well and fail to resolve clusters.\n\nI don't agree the claim presented in the lines 38-40 about entity coreference resolution. Semantic matching is only one of aspects need to be considered in the coreference resolution task. Maybe two mentions have identical semantic meaning, but they could refer to totally different entities. In addition, coreference resolution system also needs to consider compatibilities, such as agreement of number, gender and entity type.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper proposes a prompt-based approach for the event coreference resolution (ECR) task, considering compatibilities of event type and argument. The authors conduct an analysis comparing prompt tuning and fine-tuning to show the advantages of prompt-based method on the ECR task.", "excitement": 2, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviews have not reached a consensus on both soundness and excitement.

Regarding soundness, there is a discrepancy in whether the empirical observations (especially Table 1) adequately support the effectiveness of the proposed method.  I see the effectiveness is well verified, because the proposed method uses segment-level contexts and outperforms state-of-the-art  (Lu and Ng, 2021b) using segment-level contexts, while rivaling state-of-the-art (Xu et al., 2022) using document-level contexts.

As for excitement, the proposed prompt-based method for within-document event coreference resolution is novel.  I would value this novelty, because to my knowledge this paper is the first paper that employs prompting techniques for event coference resolution.
Decision: Accept-Main