Unified Generative Model with Multi-Dimensional Prefix for Zero-Shot Event-Relational Reasoning
Year: 2023

Abstract:
Reasoning about events and their relations attracts surging research efforts since it is regarded as an indispensable ability to fulfill various event-centric or common-sense reasoning tasks. However, these tasks often suffer from limited data availability due to the laborintensive nature of their annotations. Consequently, recent studies have explored knowledge transfer approaches within a multi-task learning framework to address this challenge. Although such methods have achieved acceptable results, such brute-force solutions struggle to effectively transfer event-relational knowledge due to the vast array of inter-event relations (e.g. temporal, causal, conditional) and reasoning formulations (e.g. discriminative, abductive, ending prediction). To enhance knowledge transfer and enable zero-shot generalization among various combinations, in this work we propose a novel unified framework, called UNIEVENT. Inspired by prefix-based multitask learning, our approach organizes event relational reasoning tasks into a coordinate system with multiple axes, representing inter-event relations and reasoning formulations. We then train a unified text-to-text generative model that utilizes coordinate-assigning prefixes for each task. By leveraging our adapted prefixes, our unified model achieves state-of-the-art or competitive performance on both zero-shot and supervised reasoning tasks, as demonstrated in extensive experiments.

heading: 1 Introduction
text: An ‘event’ is defined as a semantic molecule to explain the states or actions of a person, entity, or thing (Zhou et al., 2022). In natural language literature, it is usually represented as a span in
∗*Corresponding author.
narrative text (e.g., sentences, paragraphs or documents), which is composed of an event trigger (e.g., predicate) and its arguments (e.g., subject, object, adverbial modifier). Based on the semantic unit at the event level, a broad spectrum of eventrelational reasoning tasks was presented to learn various inter-event relations (e.g., temporal, causal, conditional) and thus enable commonsense or cognitive reasoning capabilities for advanced AI systems. The inherent event-relational reasoning logic has been formulated as tasks as relation extraction (Han et al., 2021b), question answering (Yang et al., 2022b; Han et al., 2021a), intend prediction (Rashkin et al., 2018), summarization (Daumé and Marcu, 2006) and knowledge base construc-
7088
tion (Sap et al., 2019; Li et al., 2020).
Attributed to recently advanced language models (e.g., BERT and GPT-3) pre-trained on raw corpora with billions of words in a self-supervised manner, data-driven methods via a pre-training & fine-tuning paradigm achieves acceptable performance on the event-relational reasoning tasks (Han et al., 2021b; Chen et al., 2022; Man et al., 2022a). Nonetheless, its inherently complex intra-event semantics and intricate inter-event relations inevitably increase the labor intensity of human annotation processes (e.g., experts-required, timeconsuming, label-inconsistent). This limits the scale of human-labeled data for fine-tuning and thus affects the effectiveness of the data-driven methods on those tasks (Ning et al., 2018). For example, considering the event temporal question answering task, there are only 198 training instances in CIDER (Ghosal et al., 2021) among all datasets.
Therefore, such a data-scarcity issue necessitates knowledge transfer to an event-relational reasoning task. Besides task-specific heuristic pseudo labeling in a self- or semi-supervised framework to transfer from large-scale in-domain raw corpus, recent event-centric research works resort to supervised knowledge transfer due to its general learning methodology and superior fine-tuning performance. That is, transferring knowledge among supervised datasets under a variety of inter-event relations (e.g., temporal, causal) (Han et al., 2019; Wang et al., 2020) and reasoning formulations (e.g., event relation extraction, question answering) (Tang et al., 2021; Li et al., 2022b; Lourie et al., 2021). Despite their superior transfer performance, as shown in Figure 1, these works do not well consider knowledge transfer among a variety of both targeted relations and reasoning formulations in event-relational reasoning, and they usually fail to generalize to unseen event-relational reasoning tasks with distinct relations and/or formulations. For example, according to our empirical study shown in NT column of Table 3, unified training on T5 fails to transfer to tasks both unseen in formulation and relation.
To enhance knowledge transfer and empower zero-shot generalization among event-relational reasoning tasks, in this work we propose a brandnew unified framework UNIEVENT for zero-shot event-relational reasoning tasks transferring. We first categorize all event-relational reasoning tasks according to their original formulation types and
event relation. We then construct generative formats for each task and convert them into generation forms. We train on adapted tasks based on a pretrained generation model (Raffel et al., 2020). Based on that, the proposed unified model enables implicit transfer across event-relational reasoning tasks. However, without explicitly discriminating the categorical coordination of the data, straightforward multi-task training may suffer from negative transfer (Liu et al., 2019) and intensive diversity of formulations and relations. Therefore, inspired by recent success of prompt tuning (Lester et al., 2021; Li and Liang, 2021; Liu et al., 2021b) where prompt instruction show great benefit in multi-task training (Sanh et al., 2021; Wei et al., 2021; Xu et al., 2022; Raffel et al., 2020), we propose to add prefix (Li and Liang, 2021) adapting to diversified formulations and relations. This multi-dimensional prefix additionally facilitates further transfer across tasks. We introduce to generate of these prefixes via the Adaptive Prefix Generators which allow sharing of flexible features among distinct dimensions. We then perform a contrastive regularization (Wu et al., 2020; Su et al., 2021) to learn to discriminate various task formulations and relations and enhance the representation.
We conduct extensive experiments on 16 datasets (3 for multi-task training, 13 for testing). Experiment results demonstrate that our method shows significant transferability and outperforms baselines in both zero-shot and full data multi-task settings We summarize our contributions as:
• We propose UNIEVENT for zero-shot eventrelational reasoning tasks. We first categorize the event-relational reasoning tasks by the task formulation type and event relation. Then we unify the training datasets with eventrelational reasoning targeted generative formats to enable knowledge transfer. • We propose the Adaptive Prefix Generator to generate prefixes to guide the event-relational reasoning process. We also put up with a formulation- and relation-aware contrastive regularization to enhance further knowledge transfer across relations and formulations. • We conduct extensive experiments to testify to our method. UNIEVENT outperforms the baselines on average of all datasets in both zero-shot and full-data settings.

heading: 2 Method
text: Task Formulation. The objective of our study is to train a model using a combination of training datasets from different task formulations and event relations, enabling it to transfer its learning to a set of unseen datasets that were withheld during training. Formally, given a unified training dataset T = ⋃ Ti, we aim to train a model P (Y|X ) on T. Each data (X ,Y, ς) ∈ T consists of an input X , an label Y and the original task formulation type ς . In summary, our framework encompasses relation extraction, natural language inference, question answering, and multiple-choice formulations., i.e. ς ∈ {RE,NLI,QA,MC}. For all types of formulation, the inputs X and label Y are specifically:
(X ,Y) =    ((D, E0, E1, γ), L), ς = RE ((D, γ), L), ς = NLI ((D, Q, γ), A), ς = QA ((D, E0, E1, I, γ), A), ς = MC
where D indicates the document, E0 and E1 are two queried events, Q is a question about events, I stands for the queried dimension (e.g. cause and result for event causality tasks), γ denotes for inherent event relation of that data, L is the gold label and A is the gold answer text. Then we transfer the models to the held-out unseen datasets Z = ⋃ Zz which is also composed by such four types of tasks. In this paper, we mainly consider four event relations which are temporal (TEMP), causal (CA), counterfactual (COUNT), and conditional (COND). We finally result in tasks taxonomy as shown in Table 1.
Model Overview. Our model undergoes training on unified diverse datasets of task formulations and event relations, followed by evaluation on held-out test sets where it encounters zero-shot scenarios. We first convert all tasks into text-to-text generation based on our constructed generative formats as in Section 2.1. After that, UNIEVENT takes input with multi-dimensional prefix concatenated and generates output sequence. To improve knowledge transfer, we use the Adaptive Prefix Generators to generate the above prefixes according to the formulation and the relation of each data as in Section 2.2 and propose the formulation- and relation-aware contrastive regularizationas in Section 2.3. Finally, UNIEVENT perform unified multi-task training in Section 2.4. We depict an overview of UNIEVENT as shown in Figure 2.

heading: 2.1 Unified Generative Adaptation
text: We adapt all tasks into generation forms with constructed generative formats to enable unified generative training. However, there have been no available human-engineered prompts for eventrelated tasks so far. As is known to all, model performance is sensitive to the prompt and verbalizer designs (Shin et al., 2020). Such prompts from Prompt Source1 are not directly suitable for event-relational reasoning tasks. Considering that, we construct the discrete generative formats from scratch. The generative format varies with task formulations and event relations, as listed in Figure 2. We mainly take RE as an example to explain the following process.
Input Adaptation. The adapted input is mainly a question "What is the relation between E0 and E1 ?". E0 and E1 are queried events. We prepend the document content placeholder D before the question. We also append optional O representing the candidate label set. For MC, there’s another placeholder I which denotes for queried dimension(eg. cause and effect for the causal relation).
Ouput Adaptation. Conventionally in prompt tuning (Shin et al., 2020), we construct a verbalizer VERB(·) to map relation labels L to label words. As is shown in Figure 2, we show all verbalizers for all mentioned event relations. After, the generation output is the label word VERB(L) appended by the [eos] indicator. In QA and MC, we directly take the original answer A to compose the output. As a result, for data of any formulation and relation, we convert it into a text-to-text form with input X and linearized output sequence Y .
Model Generation. Then given an input X , X = (x1, x2, ..., xn) where xi is the ith token of the input X and n is the sequence length, UNIEVENT output the prediction by generating the linearized answer Y . The generation process is modeled by a pretrained encoder-decoder language model M such as BART (Lewis et al., 2019) and T5 (Raffel et al., 2020) which are pretrained on a largescale corpus. After the generation adaptation, UNIEVENT first encode X by the encoder Enc of M. Each encoder layer of M is a multi-head self-attention (Vaswani et al., 2017) block which take H l ∈ Rn×d as input to compute input of next layer H l+1 = Encl(H l; θle). d is the hidden state
1https://github.com/bigscience-workshop/promptsource
dimension. UNIEVENT then generate answer Y with decoder of M in an auto-regressive generation process. We use θM = (θe, θd) to denote both encoder and decoder parameters of M
P (Y|X ) = ∏
i
Dec(Y<i,H; θd). (1)

heading: 2.2 Multi-Dimensional Prefix-Tuning
text: Straightforward unifying all tasks can impede a model’s ability to recognize distinct formulations and relations, and could further result in negative transfer (Liu et al., 2019). To have UNIEVENT adapt to different tasks and relations while share basic information across them, we propose to use multi-dimensional prefix to instruct the generation. We generate formulation-wise prefix matrix Pkς and relation-wise prefix matrix Pkγ via our Adaptive Prefix Generators. To further train the Adaptive Prefix Generators and facilitate the discriminated representation, we propose the Task- and Relationaware Contrastive Regularization.

heading: 2.2.1 Adaptive Prefix Generator (APG)
text: To better adapt UNIEVENT to different formulation types and relations, we utilize prepended layer-wise prefixes (Li and Liang, 2021) to guide the generation. Moreover, on account of sharing flexible features of various task formulations and event relations, we instead generate these prefixes via a novel Adaptive Prefix Generators.
We first introduce the learnable embeddings V lk ∈ Rs×d p for various aspects in each layer, k ∈ A. A can be any considering attributes which in this paper is the set of task formulations or event relations. dp is the vector dimension, s is the length. l ∈ [1, L] is the layer index. V lk can be randomly initialized or pretrained from other tasks before.
Given V lk , our APG g l(·) takes it as input and generates dimension-specific prefix P lk. g l(·) consists a trainable bottleneck layer which is a pair of down and up projections that firstly align different knowledge representations to the same semantic space and then project them to space of M. Mathematically, given V lk
P lk[i, :] := g l(V lk [i, :]; θg)
=W u l Tanh (W d l V lk [i, :]),
P lk[i, :] ∈ Rd, i ∈ [1, s], (2)
where W d l ∈ Rdp×dm and W ul ∈ Rdm×d. dm is the mid dimension of the bottleneck layer. Tanh
is the hyperbolic tangent activation function. θg is the parameter for APG. The APG can apply to the both formulation and relation axis. Specifically, for formulation-wise APG, the attributes set A is:
A = {RE,NLI,QA,MC}.
Turning to event relation:
A = {TEMP,CA,COUNT,COND}.
The Adaptive Prefix Generator are learned end-toend with the backbone transformer M.

heading: 2.2.2 Prefix Instructed Generation
text: To instruct the accomplishment of a task and induce considering task formulation and relational knowledge from the model. We prepend generated formulation-wise and relation-wise prefix matrix P lkς and P l kγ to inputs of each encoder layer of M:
H l+1 = Encl([P lkς ;P l kγ ;H l]; θle), (3)
H l ∈ Rn×d,P lkς ∈ Rs ς×d,P lkγ ∈ Rs γ×d,
H is hidden states of the lth layer. sς and sγ are lengths of formulation-wise and relation-wise prefix respectively. [; ] is the concatenation operation.
We also add non-identical prefixes generated by another two APGs to each layer of decoder. Therefore, in total, we have four APGs in UNIEVENT.

heading: 2.3 Formulation- and Relation-aware Contrastive Regularization (TRC)
text: When trained solely on the supervised multi-task loss, a model has a tendency to undergo shortcuts of neglecting the prefixes. If this happens, UNIEVENT degrades to normal multi-task training on M. To avoid such a dilemma and further adapt UNIEVENT
to various dimensions, we add an additional contrastive regularization (Wu et al., 2020; Su et al., 2021). We take the vector H[bos] of the first token [bos] after all prefixes from the last layer’s hidden states as the representation. Then we map H[bos] to another space via a feed-forward layer f(·): uX := f(H[bos]; θc) = Tanh(W cH[bos]), (4)
uX ∈ Rd. W c ∈ Rd×d c . θc represents the parameters. Then we take uX as the representation of X . For a data point X with its formulation type ςX and event relation γX , we sample a subset KX from the whole training set. Then we conduct contrastive learning on X with KX :
φX = ∑
Xp∈ K+X
log exp(uX · uXp/τ)∑
Xa∈ KX exp(uX · uXa/τ)
,
(5)
LC = − ∑
X∈ T
1
|KX | φX ,
K+X = {Xp|Xp ∈ KX , ςXp = ςX ∧ γXp = γX }, where τ is the temperature parameter and · is vector inner production.

heading: 2.4 Multi-Task Training
text: To train UNIEVENT, we perform multi-task training on T. We shuffle all data of T which ends up with a mixed-up training batch composed of data from various datasets. Then we acquire the final training loss with scaling factor α:
LE = − ∑
(X ,Y)∈ T logP (Y|X ; θM, θg, θc),
L = LE + α× LC , (6)

heading: 3 Experiments

heading: 3.1 Event-Relational Reasoning Datasets
text: In total, we assess the performance of UNIEVENT across 16 datasets that involve event-relational reasoning . Datasets can be divided by their original formulation types and event relation. Datasets we use are TB-Dense (TBD) (Chambers et al., 2014), MATRES (MA) (Ning et al., 2018), RED (O’Gorman et al., 2016), TDDMan (TM) (Naik et al., 2019) which are temporal relation extraction; ESL (Caselli and Vossen, 2017)2, SCITE (SCI) (Li et al., 2021), CTB (CTB) (Mirza and Tonelli, 2016) which are event causality identification; CNC (CNC) (Tan et al., 2022b), ALTLEX (ALT) (Liang et al., 2022) which are causal natural language inference; ESTER (EST) (Lester et al., 2021), CQA (Yang et al., 2022c) and CIDER (CID) (Ghosal et al., 2021) are multi-relational question extractive answering datasets which cover causal, counterfactual and conditional event relation. RECCON-IE (RI) and RECCON-DD (RD) (Poria et al., 2021) are causal QA tasks. SE2020-EQA (SE) (Yang et al., 2020) which is a counterfactual question answering task. ECARE (ECA) (Du et al., 2022) is a causal multiple choice task. To better show the results, in the following part, we organize RD, RI, SE, CQA, CID as QA part and leave the rest as NLU part. We summarize data statistics in Figure 8 and state the details of each dataset in Appendix A. We select TBD, CNC, and EST as train sets and leave others
2In this paper, we don’t perform 5-folds cross-validation and instead split each dataset into 8 :1 :1 for training, validation and test.
as held-out unseen test datasets.

heading: 3.2 Evaluation Metrics
text: All evaluation metrics follow previous researches on each dataset. We use micro-F1 score to evaluate all relation extraction tasks. Since causal NLI only has two labels(entailment and contradiction), we evaluate them by binary-F1 score. We denote both micro-F1 and binary-F1 as F1. We use F1score (F1), EM to measure QA task. F1 measures the correctness of uni-grams in generated sentence comparing those in ground truth sentences. EM score measures the exactly matches of uni-grams. In ESTER dataset, previous works also evaluate by HIT@1 which measures whether the event trigger words are generated in the sentences. Multiple choice tasks are measured by accuracy.

heading: 3.3 Parameters
text: We choose T5-base (Raffel et al., 2020) as the backbone of UNIEVENT. We set both formulation-wise and relational knowledge-wise prefix length sς and sγ as 200. For all experiments, we use batch size 32, learning rate 5e-5 on AdamW optimizer. For contrastive learning, we set τ = 0.07, α = 0.05 and |K| = 512. We don’t use any optimization tricks like label smoothing and randomly initialize all parameters our Adaptive Prefix Generators. We train till 15 epochs for all model and select best performing checkpoint on average score of all validation sets. We use deepspeed3 framework and train on two Tesla V-100 GPUs.
3https://www.deepspeed.ai/

heading: 3.4 Baselines
text: • T0-3B(Sanh et al., 2021) This is the strongest baseline which is trained on a massive corpus of hundreds of general datasets. And more, this model is 10× bigger than our model. • T5-zero (Raffel et al., 2020). We directly test on T5 without any training. • T5-unified This is the baseline that only conducts multi-task training on T5-base without multi-dimensional prefix-tuning. • UniEvent-r This is the ablated model of UNIEVENT without relation-wise prefixes. • UniEvent-t This is the ablated model of UNIEVENT without formulation-wise prefixes. • UniEvent-c This is the ablated model of UNIEVENT without formulation- and relationaware contrastive regularization.

heading: 3.5 Zero-Shot Results
text: We list models’ average performances on all zeroshot test datasets in Table 3. We find UNIEVENT outperforms strong baseline T0-3B on average 6.52 scores of all tasks in column ALL. This demonstrate the effectiveness of transferability on zeroshot event-relational reasoning tasks. The multidimensional prefixes with task- and relation-aware contrastive loss further boost the model to transfer across tasks. We also find T5-unified achieves comparable performance with T0-3B which is 10 × larger than it. All above findings testify our motivations that transfer knowledge via task formulation and relation axis is promising. Moreover, our multidimensional prefix-tuning ensures the knowledge
transfer.
We list average score of QA tasks of all models in columns QA-F1 (i.e. average of f1-scores.), QA-Em (i.e. average of exactly match scores.) and QA of Table 3 and show score of each dataset in Table 4. In Table 3, we find UNIEVENT outperforms T0-3B with average 9.43 scores on QA which is average scores of all both F1 and EM. This reveals UNIEVENT works encouragingly on QA reasoning. We show average score of NLU tasks in column NLU of Table 3 and results of each dataset in Table 5. We find UNIEVENT exceeds 0.48 scores on average which indicates the effectiveness of UNIEVENT on NLU part of datasets. As we can find, UNIEVENT performs not that well on NLU as on QA, we believe this is probably due to the pretrained generation backbone M is more suitable for generation tasks and T0-3B are trained on massive NLU datasets.
We also conduct experiments to evaluate crossformulation and cross-relation transfer. Results are listed in NT column in Table 3 which are average scores of all datasets without training data in the same coordination in Figure 1. We surprisingly find that UNIEVENT exceeds T0-3B on a large margin, i.e. 8.12 average scores. These results indicate promising transferability of UNIEVENT since those tested dataset can only be completed by transfering from other datasets.
We report performances on TEMP datasets (MA, RED, TM, temporal part of CID) and CA datasets(ESL, SCI, CTB, ALT, ECA, RD, RI, causality parts of CQA and CID) of all models as well. Results are illustrated in TP and CA columns
in Table 3. Firstly, we find UNIEVENT performs well on CA datasets. However, we find formulationwise prefixes harms performances of TEMP tasks which is probably due to most of TEMP datasets are RE.

heading: 3.6 Multi-Task Training Results
text: We also report multi-task training results on three trainsets. We find scores of trainsets can still increase if we continue training after the 10th epoch while zero-shot performance would drop. Therefore, for fair comparison, we report best results within 10th epochs for all models. As shown in the Table 6, UNIEVENT exceeds T5-unified. T5-base is a model finetuned on T5 base model in single task. Results demonstrate that our unified model can even transfer knowledge in full data setting. We believe our multi-dimensional prefix-tuning can reduce notorious negative transfer to some degree.

heading: 3.7 Ablation Study
text: Model Ablation. We conduct model ablation studies. The results are detailed in Table 3. We find both formulation-wise and relation-wise prefixes effect. UniEvent outperforms UniEvent-c, which indicates task- and relation-aware contrastive regularization is crucial since it discriminates all sorts of dimensions in the unified training.
Dataset Ablation. In order to inspect the transferability and quantify the amount, we conduct dataset ablation studies. We complete three experiments, each with one of three training set ablated. Then we compute the transfer ratio of each trainset on all metrics as x−x̂x , where x is the score of UniEvent, x̂ is the score with a dataset ablated. We detail the results in Figure 4. Basically, these experimental results are consistent with our motivation. EST contributes to all QA datasets. Causal part of EST transfer to CTB. CNC transfers causality knowledge to SCI, ESL, CTB, ALT and QA datasets as RD and RI. TBD can transfer to most of the RE dataset except MA. We believe MA suffers from negative transfer of all training sets. We surprisingly find TBD contributes to RD and RI. In sum, all training sets can transfer to other datasets on average (AVG row of Figure 4).

heading: 3.8 Prefix Length
text: In this part, we study influence of prefix length. In UniEvent, there are two types of prefix, i.e. Pkς and Pkγ . We illustrate the results in Figure 3. Specifically, in Figure 3(a), we fix length of Pkγ to 200, and vary length of Pkς (i.e. sς ) from 50 to 400. We find almost all average metrics increase with sς varying from 50 to 400 except from temporal relation average performance. The results show that formulation-wise prefix length should reach to a scale to guarantee zero-shot performance.
On the other hand, we also analysis the length of Pkγ under fixed sς = 200. Results are depicted in Figure 3(b). Results are similar with sς , sγ should reach a critical scale to make Pkγ work.
We also find a interesting phenomena that NT metrics are still increasing in both experiments which indicates prefix length should be large for both formulation and relation unseen tasks.

heading: 3.9 Dataset Substitution
text: We substitute training set with TM, ALT and CQA. Results are shown in Table 7. We find UNIEVENT outperforms all baselines with dataset substituted. It indicates that UNIEVENT can transfer knowledge in various datasets permutations.

heading: 4 Related Work
text: Unified Training To fulfill knowledge transfer, sorts of brute-force solutions known as multitask learning trains parameter-sharing neural models (Raffel et al., 2020; Sanh et al., 2021; Xu et al., 2022; Wei et al., 2021; Li et al., 2022a). However, learning out-of-domain and -formulation data could diminish the model efficacy on the targeted tasks, not to mention domain/formulation varying significantly in event-relational reasoning. Built upon a multi-task learning framework recent works are dedicated to integrating knowledge by unifying massive tasks (Lourie et al., 2021; Zhong et al., 2022; Xie et al., 2022; Lu et al., 2022; Khashabi et al., 2020). Via unified task formulations (e.g., text-to-text generation) and advanced training strategies, these works excel single task finetuning in conventional multi-task learning. Prompting Transfer Yang et al. (2022a); Liu et al. (2022); Gu et al. (2022); Asai et al. (2022); Vu et al. (2021) transfer knowledge from pretrained tasks to downstream ones via prompting. In this work, we don’t acquire prior knowledge from other tasks while enhance generalization across tasks.
Event-Relational Reasoning Zuo et al. (2020); Liu et al. (2021a); Zuo et al. (2021a); Cao et al. (2021); Zuo et al. (2021b); Chen et al. (2022); Phu and Nguyen (2021); Man et al. (2022b) identify event causality between two event trigger mentions. Zuo et al. (2020); Liu et al. (2021a); Zuo et al. (2021a) utilize external knowledge. Chen et al. (2022); Phu and Nguyen (2021) develop novel graph neural networks to capture structural information. Tan et al. (2022b); Liang et al. (2022) obtain event causality via natural language inference formulation.
Mathur et al. (2021); Zhou et al. (2020, 2021); Han et al. (2021b); Zhang et al. (2021); Hwang et al. (2022); Man et al. (2022a) extract temporal relations of events from documents or sentences. Zhou et al. (2020, 2021); Han et al. (2021b) learn from unsupervised or distant supervision.
Yang et al. (2020) asks for counterfactual statements. Du et al. (2022) aims to choose correct cause or effect from choices. Poria et al. (2021); Han et al. (2021a); Yang et al. (2022c) question about diversified event relations. Among all methods, we are the first to study the unification across these relations and formulations.

heading: 5 Conclusion
text: In this work, we propose UNIEVENT to transfer knowledge for unseen event-relational reasoning tasks. We first categorize these tasks. Then we construct generative formats and then unify them with generated multi-dimensional prefixes. UNIEVENT outperforms all baselines in both zero-shot and fulldata settings.

heading: 6 Acknowledgement
text: Our work is supported by the National Key Research and Development Program of China (Project Number: 2020AAA0109400). we kindly appreciate all the researchers who provide valuable insights, discussions, and comments on this work.
Limilations
The current UniEvent is limited to performing event-relational reasoning tasks in a textual modality. It is unable to transfer knowledge between tasks of different modalities. However, combining event knowledge from different modalities may have more interactions and further enhance performance. As this is beyond the scope of our current work, we leave it to future research.

heading: A Dataset Details
text: In this section, we state processing details of all datasets. We show dataset statistics in Table 8.
Considering temporal event relation extraction, we strictly follow settings in Han et al. (2021b) for MATRES, TBD, RED and setting in (Naik et al., 2019) for TM.
For event causality identification, in ESL, CTB, we don’t perform 5-folds cross validation as in Zuo
et al. (2021b) and instead split each dataset into 8:1:1 for train, validation and test. We follow Li et al. (2021) for SCI.
We follow CNC in Tan et al. (2022b) and ALT in Liang et al. (2022) respectively for causal NLI.
In view of question answering datasets, we follow Han et al. (2021a), Yang et al. (2022c), Ghosal et al. (2021) and Yang et al. (2020) for EST, CQA, CID and SE. RD and RI are the same with Poria et al. (2021).
Lastly, the setting for ECA is the same with Du et al. (2022).
There are no validation set for SCI, RI, SE, so when compute average score in validation, we don’t consider these three datasets.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: A1. Did you describe the limitations of your work?
Left blank.
A2. Did you discuss any potential risks of your work? Left blank.
A3. Do the abstract and introduction summarize the paper’s main claims? Left blank.
A4. Have you used AI writing assistants when working on this paper? Left blank.

heading: B Did you use or create scientific artifacts?
text: Left blank.
B1. Did you cite the creators of artifacts you used? Left blank.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? Left blank.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Left blank.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Left blank.
B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Left blank.

heading: C Did you run computational experiments?
text: Left blank.
C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Left blank.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Left blank.
C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Left blank.
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Left blank.

heading: D Did you use human annotators (e.g., crowdworkers) or research with human participants?
text: Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Left blank.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Left blank.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Left blank.

heading: Future Work
text: As this is beyond the scope of our current work, we leave it to future research