SPEECH : Structured Prediction with Energy-Based Event-Centric Hyperspheres (2023)

Abstract: Event-centric structured prediction involves predicting structured outputs of events. In most NLP cases, event structures are complex with manifold dependency, and it is challenging to effectively represent these complicated structured events. To address these issues, we propose Structured Prediction with Energy-based Event-Centric Hyperspheres (SPEECH). SPEECH models complex dependency among event structured components with energy-based modeling, and represents event classes with simple but effective hyperspheres. Experiments on two unified-annotated event datasets indicate that SPEECH is predominant in event detection and event-relation extraction tasks.

## introduction structured prediction @xcite is a task where the predicted outputs are complex structured components. this arises in many nlp tasks @xcite @xcite and supports various applications @xcite . in event-centric nlp tasks, there exists strong complex dependency between the structured outputs, such as event detection (ed) @xcite , event-relation extraction (ere) @xcite , and event schema induction @xcite . thus, these tasks can also be revisited as event-centric structured prediction problems @xcite . event-centric structured prediction (ecsp) tasks require to consider manifold structures and dependency of events, including intra-/inter-sentence structures. for example, as seen in figure 1 , given a document containing some event mentions "david warren shot and killed henry glover ... david was convicted and sentenced to 25 years and 9 months ...", in ed task mainly considering intra-sentence structures, we need to identify event triggers (killed, convicted) from these tokens and categorize them into event classes (killing, legal_rulings); in ere task mainly considering inter-sentence structures, we need to find the relationship between each event mention pair, such as event coreference, temporal, causal and subevent relations. as seen from figure 1 , the outputs of ecsp lie on a complex manifold and possess interdependent structures, e.g., the long-range dependency of tokens, the association among triggers and event classes, and the dependency among event classes and event relations. thus it is challenging to model such complex event structures while efficiently representing these events. previous works increasingly apply deep representation learning to tackle these problems. @xcite ; @xcite propose to predict event structures based on the event graph schema. @xcite generate event structures with manually designed prompts. however, these methods mainly focus on one of ecsp tasks and their event structures are hard to represent effectively. @xcite ; @xcite propose to extract multiple event structures from texts with a unified generation paradigm. however, the event structures of these approaches are usually quite simplistic and they often ignore the complex dependency among tasks. in this paper, we focus more on: (i) how to learn complex event structures for manifold ecsp tasks; and (ii) how to simultane-ously represent events for these complex structured prediction models effectively. to resolve the first challenging problem of modeling manifold event structures, we utilize energy networks @xcite @xcite , inspired by their potential benefits in capturing complex dependency of structured components. we define the energy function to evaluate compatibility of input/output pairs, which places no limits on the size of the structured components, making it powerful to model complex and manifold event structures. we generally consider token-, sentence-, and document-level energy respectively for trigger classification, event classification and event-relation extraction tasks. to the best of our knowledge, this work firstly address event-centric structured prediction with energy-based modeling. to resolve the second challenging problem of efficiently representing events, we take advantage of hyperspheres @xcite , which is demonstrated to be a simple and effective approach to model class representation @xcite . we assume that the event mentions of each event class distribute on the corresponding energy-based hypersphere, so that we can represent each event class with a hyperspherical centroid and radius embedding. the geometrical modeling strategy @xcite is demonstrated to be beneficial for modelling enriched class-level information and suitable for constructing measurements in euclidean space, making it intuitively applicable to manifold eventcentric structured prediction tasks. summarily, considering the two issues, we propose to address structured prediction with energybased event-centric hyperspheres (speech), and our contributions can be summarized as follows: ## experiments the experiments refer to event-centric structured prediction (ecsp) and comprise three tasks: (1) trigger classification; (2) event classification; and ## conclusion and future work in this paper, we propose a novel approach entitled speech to tackle event-centric structured prediction with energy-based hyperspheres. we represent event classes as hyperspheres with token, sentence and document-level energy, respectively for trigger classification, event classification and event relation extraction tasks. we evaluate speech on two event-centric structured prediction datasets, and experimental results demonstrate that speech is able to model manifold event structures with dependency and obtain effective event representations. in the future, we intend to enhance our work by modeling more complicated structures and extend it to other structured prediction tasks.