SPEECH: Structured Prediction with Energy-Based Event-Centric Hyperspheres
Year: 2023

Abstract:
Event-centric structured prediction involves predicting structured outputs of events. In most NLP cases, event structures are complex with manifold dependency, and it is challenging to effectively represent these complicated structured events. To address these issues, we propose Structured Prediction with Energybased Event-Centric Hyperspheres (SPEECH). SPEECH models complex dependency among event structured components with energybased modeling, and represents event classes with simple but effective hyperspheres. Experiments on two unified-annotated event datasets indicate that SPEECH is predominant in event detection and event-relation extraction tasks.

heading: 1 Introduction
text: Structured prediction (Taskar et al., 2005) is a task where the predicted outputs are complex structured components. This arises in many NLP tasks (Smith, 2011; Kreutzer et al., 2017; Wang et al., 2023) and supports various applications (Jagannatha and Yu, 2016; Kreutzer et al., 2021). In event-centric NLP tasks, there exists strong complex dependency between the structured outputs, such as event detection (ED) (Chen et al., 2015), event-relation extraction (ERE) (Liu et al., 2020b), and event schema induction (Li et al., 2020). Thus, these tasks can also be revisited as event-centric structured prediction problems (Li et al., 2013).
Event-centric structured prediction (ECSP) tasks require to consider manifold structures and dependency of events, including intra-/inter-sentence structures. For example, as seen in Figure 1, given a document containing some event mentions “David Warren shot and killed Henry Glover ... David was convicted and sentenced to 25 years and 9 months ...”, in ED task mainly considering intra-sentence structures, we need to identify event triggers (killed, convicted) from these tokens and categorize them
∗ Corresponding Author.
into event classes (killing, legal_rulings); in ERE task mainly considering inter-sentence structures, we need to find the relationship between each event mention pair, such as event coreference, temporal, causal and subevent relations.
As seen from Figure 1, the outputs of ECSP lie on a complex manifold and possess interdependent structures, e.g., the long-range dependency of tokens, the association among triggers and event classes, and the dependency among event classes and event relations. Thus it is challenging to model such complex event structures while efficiently representing these events. Previous works increasingly apply deep representation learning to tackle these problems. Lin et al. (2020); Li et al. (2020) propose to predict event structures based on the event graph schema. Hsu et al. (2022) generate event structures with manually designed prompts. However, these methods mainly focus on one of ECSP tasks and their event structures are hard to represent effectively. Paolini et al. (2021); Lu et al. (2021, 2022) propose to extract multiple event structures from texts with a unified generation paradigm. However, the event structures of these approaches are usually quite simplistic and they often ignore the complex dependency among tasks. In this paper, we focus more on: (i) how to learn complex event structures for manifold ECSP tasks; and (ii) how to simultane-
351
ously represent events for these complex structured prediction models effectively.
To resolve the first challenging problem of modeling manifold event structures, we utilize energy networks (Lecun et al., 2006; Belanger and McCallum, 2016; Belanger et al., 2017; Tu and Gimpel, 2018), inspired by their potential benefits in capturing complex dependency of structured components. We define the energy function to evaluate compatibility of input/output pairs, which places no limits on the size of the structured components, making it powerful to model complex and manifold event structures. We generally consider token-, sentence-, and document- level energy respectively for trigger classification, event classification and event-relation extraction tasks. To the best of our knowledge, this work firstly address event-centric structured prediction with energy-based modeling.
To resolve the second challenging problem of efficiently representing events, we take advantage of hyperspheres (Mettes et al., 2019; Wang and Isola, 2020), which is demonstrated to be a simple and effective approach to model class representation (Deng et al., 2022). We assume that the event mentions of each event class distribute on the corresponding energy-based hypersphere, so that we can represent each event class with a hyperspherical centroid and radius embedding. The geometrical modeling strategy (Ding et al., 2021; Lai et al., 2021) is demonstrated to be beneficial for modelling enriched class-level information and suitable for constructing measurements in Euclidean space, making it intuitively applicable to manifold eventcentric structured prediction tasks.
Summarily, considering the two issues, we propose to address Structured Prediction with Energybased Event-Centric Hyperspheres (SPEECH), and our contributions can be summarized as follows:
• We revisit the event-centric structured prediction tasks in consideration of both complex event structures with manifold dependency and efficient representation of events.
• We propose a novel approach named SPEECH to model complex event structures with energy-based networks and efficiently represent events with event-centric hyperspheres.
• We evaluate SPEECH on two newly proposed datasets for both event detection and eventrelation extraction, and experiments demonstrate that our model is advantageous.

heading: 2 Related Work
text: Event-Centric Structured Prediction (ECSP). Since the boom in deep learning, traditional approaches to ECSP mostly define a score function between inputs and outputs based on a neural network, such as CNN (Chen et al., 2015; Deng et al., 2020), RNN (Nguyen et al., 2016; Meng and Rumshisky, 2018; Nguyen and Nguyen, 2019), and GCN (Yan et al., 2019; Lai et al., 2020; Cui et al., 2020). With the development of pretrained large models, more recent research has entered a new era. Wang et al. (2019); Du and Cardie (2020); Liu et al. (2020a); Deng et al. (2021); Sheng et al. (2022) leverage BERT (Devlin et al., 2019) for event extraction. Han et al. (2020) and Wang et al. (2020a); Man et al. (2022); Hwang et al. (2022) respectively adopt BERT and RoBERTa (Liu et al., 2019) for event-relation extraction. Lu et al. (2021); Paolini et al. (2021); Lu et al. (2022) propose generative ECSP models based on pre-trained T5 (Raffel et al., 2020). Wang et al. (2023) tackle ECSP with code generation based on code pretraining. However, these approaches are equipped with fairly simplistic event structures and have difficulty in tackling complex dependency in events. Besides, most of them fail to represent manifold events effectively.
Energy Networks for Structured Prediction and Hyperspheres for Class Representation. Energy networks define an energy function over input/output pairs with arbitrary neural networks, which places no limits on the size of the structured components, making it advantageous in modeling complex and manifold event structures. Lecun et al. (2006); Belanger and McCallum (2016) associate a scalar measure to evaluate the compatibility to each configuration of inputs and outputs. (Belanger and McCallum, 2016) formulate deep energy-based models for structured prediction, called structured prediction energy networks (SPENs). Belanger et al. (2017) present end-to-end learning for SPENs, Tu and Gimpel (2018) jointly train structured energy functions and inference networks with largemargin objectives. Some previous researches also regard event-centric NLP tasks as structured prediction (Li et al., 2013; Paolini et al., 2021). Furthermore, to effectively obtain event representations, Deng et al. (2022) demonstrate that hyperspherical prototypical networks (Mettes et al., 2019) are powerful to encode enriched semantics and dependency in event structures, but they merely consider support for pairwise event structures.

heading: 3 Methodology

heading: 3.1 Preliminaries
text: For structured prediction tasks, given input x ∈ X , we denote the structured outputs by MΦ(x) ∈ Ỹ with a prediction model MΦ. Structured Prediction Energy Networks (SPENs) score structured outputs with an energy function EΘ : X ×Ỹ → R parameterized by Θ that iteratively optimize the energy between the input/output pair (Belanger and McCallum, 2016), where lower energy means greater compatibility between the pair.
We introduce event-centric structured prediction (ECSP) following the similar setting as SPENs for multi-label classification and sequence labeling proposed by Tu and Gimpel (2018). Given a feature vector x belonging to one of T labels, the model output is MΦ(x) = {0, 1}T ∈ Ỹ for all x. The energy function contains two terms:
EΘ(x,y) = E local Θ (x,y) + E label Θ (y)
= T∑
i=1
yiV > i f(x) + w
>g(Wy) (1)
where ElocalΘ (x,y) = ∑T i=1 yiV > i f(x) is the sum of linear models, and yi ∈ y, Vi is a parameter vector for label i and f(x) is a multi-layer perceptron computing a feature representation for the input x; ElabelΘ (y) = w
>g(Wy) returns a scalar which quantifies the full set of labels, scoring y independent of x, thereinto, w is a parameter vector, g(·) is an elementwise non-linearity function, and W is a parameter matrix learned from data indicating the interaction between labels.
After learning the energy function, prediction minimizes energy:
ỹ = arg min y∈Ỹ EΘ(x,y) (2)
The final theoretical optimum for SPEN is denoted by:
min Θ max Φ
∑[ 4 (MΦ(xi),yi)−
EΘ (xi,MΦ(xi)) + EΘ (xi,yi) ] +
(3) where [a]+ = max(0, a), and 4(ỹ,y), often referred to “margin-rescaled” structured hinge loss, is a structured cost function that returns a nonnegative value indicating the difference between the predicted result ỹ and ground truth y.

heading: 3.2 Problem Formulation
text: In this paper, we focus on ECSP tasks of event detection (ED) and event-relation extraction (ERE). ED can be divided into trigger classification for tokens and event classification for sentences. We denote the dataset by D = {E ,R,X} containing an event class set E , a multi-faceted eventrelation set R and the event corpus X , thereinto, E = {ei | i ∈ [1, |E|]} contains |E| event classes including a None;R = {ri | i ∈ [1, |R|]} contains |R| temporal, causal, subevent and coreference relationships among event mentions including a NA event-relation; X = {Xi | i ∈ [1,K]} consists of K event mentions, where Xi is denoted as a token sequence x = {xj | j ∈ [1, L]} with maximum L tokens. For trigger classification, the goal is to predict the index t (1 ≤ t ≤ L) of the trigger xt in each token sequence x and categorize xt into a specific event class ei ∈ E . For event classification, we expect to predict the event label ei for each event mention Xi. For event-relation extraction, we require to identify the relation ri ∈ R for a pair of event mentions Ẍ〈ij〉 = (Xi,Xj).
In summary, our goal is to design an ECSP model MΦ, aiming to tackle the tasks of: (1) trigger classification: to predict the token label ỹ = MΦ(x) for the token list x; (2) event classification: to predict the event class label Ỹ = MΦ(X) for the event mention X; (3) event-relation extraction: to predict the event-relation label z̃ = MΦ(Ẍ) for the event mention pair Ẍ .

heading: 3.3 Model Overview
text: As seen in Figure 2, SPEECH combines three levels of energy: token, sentence, as well as document, and they respectively serve for three kinds of ECSP tasks: (1) token-level energy for trigger classification: considering energy-based modeling is able to capture long-range dependency among tokens without limits to token size; (2) sentencelevel energy for event classification: considering energy-based hyperspheres can model the complex event structures and represent events efficiently; and (3) document-level energy for event-relation extraction: considering energy-based modeling enables us to address the association among event mention pairs and event-relations. We leverage the trigger embeddings as event mention embeddings; the energy-based hyperspheres with a centroid and a radius as event class embeddings, and these three tasks are associative to each other.

heading: 3.4 Token-Level Energy
text: Token-level energy serves for trigger classification. Given a token sequence x = {xj |j ∈ [1, L]} with trigger xt, we leverage a pluggable backbone encoder to obtain the contextual representation f1(x) for each token, such as pre-trained BERT (Devlin et al., 2019), RoBERTa (Liu et al., 2019), DistilBERT (Sanh et al., 2019) and so on. We then predict the label ỹ = MΦ(x) of each token with an additional linear classifier. Inspired by SPENs for sequence labeling (Tu and Gimpel, 2018), we also adopt an energy function for token classification.
Energy Function. The token-level energy function is inherited from Eq (1), defined as:
EΘ(x,y) =
−   L∑
n=1
|E|+2∑
i=1
yin ( V >1,if1(xn) ) ︸ ︷︷ ︸ local + L∑ n=1 y>n−1W1yn︸ ︷︷ ︸ label
 
(4) where yin is the ith entry of the vector yn ∈ y, indicating the probability of the nth token xn being labeled with i (i for ei, |E|+1 for non-trigger and |E|+2 for padding token). f1(·) denotes the feature encoder of tokens. Here our learnable parameters are Θ = (V1,W1), thereinto, V1,i ∈ Rd is a parameter vector for token label i, and W1 ∈ R(|E|+2)×(|E|+2) contains the bilinear product between yn−1 and yn for token label pair terms.
Loss Function. The training objective for trigger classification is denoted by:
Ltok = ∑L
i=1
[ 4 (ỹi,yi)− EΘ (xi, ỹi)
+ EΘ (xi,yi) ] + + µ1LCE (ỹi,yi) (5)
where ỹi and yi respectively denote predicted results and ground truth. The first half of Eq (5) is inherited from Eq (3) for the energy function, and in the latter half, LCE (ỹi,yi) is the trigger classification cross entropy loss, and µ1 is its ratio.

heading: 3.5 Sentence-Level Energy
text: Sentence-level energy serves for event classification. Given the event mention Xi with the trigger xt, we utilize the trigger embedding f1(xt) as the event mention embedding f2(X), where f2(·) denotes the feature encoder of event mentions. We then predict the class of each event mention with energy-based hyperspheres, denoted by Ỹ = MΦ(X).
Specifically, we use an energy-based hypersphere to represent each event class, and assume that the event mentions of each event class should distribute on the corresponding hypersphere with the lowest energy. We then calculate the probability of the event mention X categorizing into the class ei with a hyperspherical measurement function:
S(X,Pi) = exp−[ ‖Pi−f2(X)‖2−γ ]+
∑|E| j=1 exp −[ ‖Pj−f2(X)‖2−γ ]+ (6)
where [a]+ = max(0, a), Pi denotes the hypersphere centroid embedding of ei. ‖ · ‖ denotes the Euclidean distance. γ is the radius of the hypersphere, which can be scalable or constant. We simply set γ = 1 in this paper, meaning that each event class is represented by a unit hypersphere. Larger S(X,Pi) signifies that the event mention X are more likely be categorized into Pi corresponding
to ei. To measure the energy score between event classes and event mentions, we also adopt an energy function for event classification.
Energy Function. The sentence-level energy function is inherited from Eq (1), defined as:
EΘ(X,Y ) =
−   |E|∑
i=1
Yi ( V >2,if2(X) ) ︸ ︷︷ ︸ local +w>2 g(W2Y )︸ ︷︷ ︸ label
  (7)
where Yi ∈ Y indicates the probability of the event mention X being categorized to ei. Here our learnable parameters are Θ = (V2, w2,W2), thereinto, V2,i ∈ Rd is a parameter vector for ei, w2 ∈ R|E| and W2 ∈ R|E|×|E|.
Loss Function. The training objective for event classification is denoted by:
Lsen = ∑K
i=1
[ 4 ( Ỹi,Yi ) − EΘ ( Xi, Ỹi )
+ EΘ (Xi,Yi) ] + + µ2LCE ( Ỹi,Yi )
(8) where the first half is inherited from Eq (3), and in the latter half, LCE is a cross entropy loss for predicted results Ỹi and ground truth Yi. µ2 is a ratio for event classification cross entropy loss.

heading: 3.6 Document-Level Energy
text: Document-level energy serves for event-relation extraction. Given event mentions X in each document, we model the embedding interactions of each event mention pair with a comprehensive feature vector f3(Ẍ〈ij〉) = [ f2(Xi), f2(Xj), f2(Xi)
f2(Xj) ] . We then predict the relation between each event mention pair with a linear classifier, denoted by z̃ = MΦ(Ẍ). Inspired by SPENs for multi-label classification (Tu and Gimpel, 2018), we also adopt an energy function for ERE.
Energy Function. The document-level energy function is inherited from Eq (1), defined as:
EΘ(Ẍ, z) =
−   |R|∑
i=1
zi ( V >3,if3(Ẍ) ) ︸ ︷︷ ︸ local +w>3 g(W3z)︸ ︷︷ ︸ label
  (9)
where zi ∈ z indicates the probability of the event mention pair Ẍ having the relation of ri. Here our learnable parameters are Θ = (V3, w3,W3), thereinto, V3,i ∈ R3d is a parameter vector for ri, w3 ∈ R|R| and W3 ∈ R|R|×|R|.
Loss Function. The training objective for eventrelation extraction is denoted by:
Ldoc = ∑N
k=1
[ 4 (z̃k, zk)− EΘ ( Ẍk, z̃k )
+ EΘ ( Ẍk, zk ) ] +
+ µ3LCE (z̃k, zk) (10)
where the first half is inherited from Eq (3), and in the latter half, LCE (z̃k, zk) is the event-relation extraction cross entropy loss, µ3 is its ratio, and N denotes the quantity of event mention pairs.
The final training loss for SPEECH MΦ parameterized by Φ is defined as:
L = λ1Ltok + λ2Lsen + λ3Ldoc + ‖Φ‖22 (11)
where λ1, λ2, λ3 are the loss ratios respectively for trigger classification, event classification and event-relation extraction tasks. We add the penalty term ‖Φ‖22 with L2 regularization.

heading: 4 Experiments
text: The experiments refer to event-centric structured prediction (ECSP) and comprise three tasks: (1) Trigger Classification; (2) Event Classification; and (3) Event-Relation Extraction.

heading: 4.1 Datasets and Baselines
text: Datasets. Considering event-centric structured prediction tasks in this paper require fine-grained annotations for events, such as labels of tokens, event mentions, and event-relations, we select two newly-proposed datasets meeting the requirements: MAVEN-ERE (Wang et al., 2022) and ONTOEVENT-DOC (Deng et al., 2021). Note that ONTOEVENT-DOC is derived from ONTOEVENT (Deng et al., 2021) which is formatted in a sentence level. We reorganize it and make it format in a document level, similar to MAVEN-ERE. Thus the train, validation, and test sets of ONTOEVENTDOC are also different from the original ONTOEVENT. We release the reconstructed dataset and
code in Github1 for reproduction. To simplify the experiment settings, we dismiss hierarchical relations of ONTOEVENT and coreference relations of MAVEN-ERE in this paper. More details of multifaceted event-relations of these two datasets are introduced in Appendix A and Github. We present the statistics about these two datasets in Table 1. The document quantity for train/valid/test set of MAVEN-ERE and ONTOEVENT are respectively 2,913/710/857, and 2,622/747/746.
Baselines. For trigger classification and event classification, we adopt models aggregated dynamic multi-pooling mechanism, i.e., DMCNN (Chen et al., 2015) and DMBERT (Wang et al., 2019); sequence labeling models with conditional random field (CRF) (Lafferty et al., 2001), i.e., BiLSTM-CRF and BERT-CRF; generative ED models, i.e., TANL (Paolini et al., 2021) and TEXT2EVENT (Lu et al., 2021). We also adopt some ED models considering document-level associations, i.e., MLBiNet (Lou et al., 2021) and CorED-BERT (Sheng et al., 2022). Besides, we compare our energy-based hyperspheres with the vanilla hyperspherical prototype network (HPN) (Mettes et al., 2019) and prototype-based model OntoED (Deng et al., 2021). Note that unlike vanilla HPN (Mettes et al., 2019) which represents all classes on one hypersphere, the HPN adopted in this paper represents each class with a distinct hypersphere. For event-relation extraction, we select RoBERTa (Liu et al., 2019), which is the same baseline used in MAVEN-ERE (Wang et al., 2022), and also serves as the backbone for most of recent ERE models (Hwang et al., 2022; Man et al., 2022).
1https://github.com/zjunlp/SPEECH.

heading: 4.2 Implementation Details
text: With regard to settings of the training process, Adam (Kingma and Ba, 2015) optimizer is used, with the learning rate of 5e-5. The maximum length L of a token sequence is 128, and the maximum quantity of event mentions in one document is set to 40 for MAVEN-ERE and 50 for ONTOEVENTDOC. The loss ratios, µ1, µ2, µ3, for token, sentence and document-level energy function are all set to 1. The value of loss ratio, λ1, λ2, λ3, for trigger classification, event classification and eventrelation extraction depends on different tasks, and we introduce them in Appendix B. We evaluate the performance of ED and ERE with micro precision (P), Recall (R) and F1 Score (F1).

heading: 4.3 Event Trigger Classification
text: We present details of event trigger classification experiment settings in Appendix B.1. As seen from the results in Table 2, SPEECH demonstrates superior performance over all baselines, notably MLBiNet (Lou et al., 2021) and CorED-BERT (Sheng et al., 2022), even if these two models consider cross-sentence semantic information or incorporate type-level and instance-level correlations. The main reason may be due to the energy-based nature of SPEECH. As seen from the last row of Table 2, the removal of energy functions from SPEECH can result in a performance decrease. Specifically for trigger classification, energy-based modeling enables capture long-range dependency of tokens and places no limits on the size of event structures. In addition, SPEECH also excels generative models, i.e., TANL (Paolini et al., 2021) and TEXT2EVENT (Lu et al., 2021), thereby demonstrating the efficacy of energy-based modeling.

heading: 4.4 Event Classification
text: The specifics of event classification experiment settings are elaborated in Appendix B.2, with results illustrated in Table 3. We can observe that SPEECH provides considerable advantages on MAVEN-ERE, while the performance on ONTOEVENT-DOC is not superior enough. ONTOEVENT-DOC contains overlapping where multiple event classes may exist in the same event mention, which could be the primary reason for SPEECH not performing well enough in this case. This impact could be exacerbated when joint training with other ECSP tasks. Upon comparison with prototype-based methods without energy-based modeling, i.e., HPN (Mettes et al., 2019) and OntoED (Deng et al., 2021), SPEECH is still dominant on MAVEN-ERE, despite HPN represents classes with hyperspheres and OntoED leverages hyperspheres integrated with eventrelation semantics. If we exclude energy functions from SPEECH, performance will degrade, as seen from the last row in Table 3. This insight suggests that energy functions contribute positively to event classification, which enable the model to directly capture complicated dependency between event mentions and event types, instead of implicitly inferring from data. Besides, SPEECH also outperforms generative models like TANL and TEXT2EVENT on MAVEN-ERE, indicating the superiority of energy-based hyperspherical modeling.

heading: 4.5 Event-Relation Extraction
text: We present the specifics of event-relation extraction experiment settings in Appendix B.3. As seen from the results in Table 4, SPEECH achieves different performance across the two ERE datasets. On ONTOEVENT-DOC dataset, SPEECH observably outperforms RoBERTa on all ERE subtasks, demonstrating the effectiveness of SPEECH equipped with energy-based hyperspheres, so that SPEECH can capture the dependency among event
mention pairs and event-relation labels. While on MAVEN-ERE, SPEECH significantly outperforms RoBERTa on ERE subtasks referring to subevent relations or trained on all event-relations, but fails to exceed RoBERTa on ERE subtasks referring to temporal and causal relations. The possible reason is that MAVEN-ERE contains less positive eventrelations than negative NA relations. Given that SPEECH models all these relations equivalently with the energy function, it becomes challenging to classify NA effectively. But this issue will be markedly improved if the quantity of positive eventrelations decreases, since SPEECH performs better on subevent relations despite MAVEN-ERE having much less subevent relations than temporal and causal ones as shown in Table 1. Furthermore, even though ONTOEVENT-DOC containing fewer positive event-relations than NA overall, SPEECH still performs well. These results suggest that SPEECH excels in modeling classes with fewer samples. Note that SPEECH also performs well when training on all event-relations (“All Joint”) of the two datasets, indicating that SPEECH is still advantageous in the scenario with more classes.

heading: 5 Further Analysis

heading: 5.1 Analysis On Energy-Based Modeling
text: We list some values of energy loss defined in Eq (5), (8) and (10) when training respectively for token, sentence and document, as presented in Figure 3. The values of token-level energy loss are observably larger than those at the sentence and document levels. This can be attributed to the fact that the energy loss is related to the quantity of samples, and a single document typically contains much more tokens than sentences or sentence pairs. All three levels of energy loss exhibit a gradual decrease over the course of training, indicating that SPEECH, through energy-based modeling, effectively minimizes the discrepancy between predicted results and ground truth. The energy functions for token, sentence and document defined in Eq (4), (7) and (9), reflect that the implementation of energy-based modeling in SPEECH is geared towards enhancing compatibility between input/output pairs. The gradually-decreasing energy loss demonstrates that SPEECH can model intricate event structures at the token, sentence, and document levels through energy-based optimization, thereby improving the outcomes of structured prediction.

heading: 5.2 Case Study: Energy-Based Hyperspheres
text: As seen in Figure 4, we visualize the event class embedding of “Attack” and 20 event mention embeddings as generated by both SPEECH and SPEECH without energy functions. We observe that for SPEECH with energy-based modelling, the instances lie near the surface of the corresponding hypersphere, while they are more scattered when not equipped with energy-based modeling, which subsequently diminishes the performance of event classification. This observation suggests that SPEECH derives significant benefits from modeling with energy-based hyperspheres. The visualiza-
tion results further demonstrate the effectiveness of SPEECH equipped with energy-based modeling.

heading: 5.3 Error Analysis
text: We further conduct error analysis by a retrospection of experimental results and datasets. (1) One typical error relates to the unbalanced data distribution. Considering every event type and event-relation contain different amount of instances, unified modeling with energy-based hyperspheres may not always be impactful. (2) The second error relates to the overlapping event mentions among event types, meaning that the same sentence may mention multiple event types. As ONTOEVENT-DOC contains many overlappings, it might be the reason for its mediocre performance on ED. (3) The third error relates to associations with event-centric structured prediction tasks. As trigger classification is closely related to event classification, wrong prediction of tokens will also influence classifying events.

heading: 6 Conclusion and Future Work
text: In this paper, we propose a novel approach entitled SPEECH to tackle event-centric structured prediction with energy-based hyperspheres. We represent event classes as hyperspheres with token, sentence and document-level energy, respectively for trigger classification, event classification and event relation extraction tasks. We evaluate SPEECH on two event-centric structured prediction datasets, and experimental results demonstrate that SPEECH is able to model manifold event structures with dependency and obtain effective event representations. In the future, we intend to enhance our work by modeling more complicated structures and extend it to other structured prediction tasks.

heading: Acknowledgements
text: We would like to express gratitude to the anonymous reviewers for their kind comments. This work was supported by the Zhejiang Provincial Natural Science Foundation of China (No. LGG22F030011), Yongjiang Talent Introduction Programme (2021A-156-G), CAAI-Huawei MindSpore Open Fund, and NUS-NCS Joint Laboratory (A-0008542-00-00).
Limitations
Although SPEECH performs well on event-centric structured prediction tasks in this paper, it still has some limitations. The first limitation relates to efficiency. As SPEECH involves many tasks and requires complex calculation, the training process is not very prompt. The second limitation relates to robustness. As seen in the experimental analysis in § 4.5, SPEECH seems not always robust to unevenly-distributed data. The third limitation relates to universality. Not all eventcentric structured prediction tasks can simultaneously achieve the best performance at the same settings of SPEECH.

heading: A Multi-Faceted Event-Relations
text: Note that MAVEN-ERE and ONTOEVENT-DOC both includes multi-faceted event-relations.
MAVEN-ERE in this paper contains 6 temporal relations: BEFORE, OVERLAP, CONTAINS, SIMULTANEOUS, BEGINS-ON, ENDS-ON; 2
causal relations: CAUSE, PRECONDITION; and 1 subevent relation: subevent_relations.
ONTOEVENT-DOC in this paper contains 3 temporal relations: BEFORE, AFTER, EQUAL; and 2 causal relations: CAUSE, CAUSEDBY.
We also add a NA relation to signify no relation between the event mention pair for the two datasets.
B Implementation Details for Different Tasks
B.1 Event Trigger Classification
Settings. We follow the similar evaluation protocol of standard ED models (Chen et al., 2015; Sheng et al., 2022) on trigger classification tasks. We present the results in Table 2 when jointly training with event classification and the whole ERE task (“All Joint” in Table 4). The backbone encoder is pretrained BERT (Devlin et al., 2019). The loss ratio, λ1, λ2, λ3 in Eq (11) are respectively set to 1, 0.1, 0.1 for both ONTOEVENT-DOC and MAVENERE.
B.2 Event Classification
Settings. We follow the similar evaluation protocol of standard ED models (Chen et al., 2015; Deng et al., 2021) on event classification tasks. We present the results in Table 3 when jointly training with trigger classification and all ERE subtasks (“+joint” in Table 4). The backbone encoder is pretrained DistilBERT (Sanh et al., 2019). The loss ratio, λ1, λ2, λ3 in Eq (11) are respectively set to 0.1, 1, 0.1 for ONTOEVENT-DOC and 1, 0.1, 0.1 for MAVEN-ERE.
B.3 Event-Relation Extraction
Settings. We follow the similar ERE experiment settings with Wang et al. (2022) on several subtasks, by separately and jointly training on temporal, causal, and subevent event-relations. We present the results in Table 4 when jointly training with trigger classification and event classification tasks. The backbone encoder is pretrained DistilBERT (Sanh et al., 2019). On ONTOEVENT-DOC dataset, the loss ratio, λ1, λ2, λ3 in Eq (11) are respectively set to 1, 0.1, 0.1 for all ERE subtasks. On MAVEN-ERE dataset, λ1, λ2, λ3 are respectively set to 0.1, 0.1, 1 for “All Joint” ERE subtasks in Table 4; 1, 1, 4 for “+joint”; 1, 0.1, 0.1 for “Temporal” and “Causal”; and 1, 0.1, 0.08 for “Subevent”.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Left blank.
A2. Did you discuss any potential risks of your work? Not applicable. Left blank.

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Abstract & at the end of Section 1 & Section 6
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 4

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 4.1

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: Section 4

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Section 4
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. I use the existing benchmark
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Not applicable. needn’t to
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 4.1
C 3 Did you run computational experiments? Section 4 Datasets and Baselines are in Section 4.1 Implementation Details are in Section 4.2 & Appendix B Main experiments are in Section 4.3, 4.4, 4.5, and Further Analysis is in Section 5
7 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? I have listed the implementation details of experiments at Sec 4.2 & Appendix B. The total computational budget & computing infrastructure used are not the main concerns of our work, and we also
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
didn’t run time statistics. But we will provide more details when publication, and the codes will also mention more details on it.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Section 4.2, Appendix B

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? We run our model and baselines multiple times and calculate an average with upper and lower bounds, which are shown in Section 4.3, 4.4, 4.5.
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Not applicable. Implementation Details are in Section 4.2 & Appendix B
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: In the future, we intend to enhance our work by modeling more complicated structures and extend it to other structured prediction tasks