TAGPRIME: A Unified Framework for Relational Structure Extraction
Year: 2023

Abstract:
Many tasks in natural language processing require the extraction of relationship information for a given condition, such as event argument extraction, relation extraction, and taskoriented semantic parsing. Recent works usually propose sophisticated models for each task independently and pay less attention to the commonality of these tasks and to have a unified framework for all the tasks. In this work, we propose to take a unified view of all these tasks and introduce TAGPRIME to address relational structure extraction problems. TAGPRIME is a sequence tagging model that appends priming words about the information of the given condition (such as an event trigger) to the input text. With the self-attention mechanism in pre-trained language models, the priming words make the output contextualized representations contain more information about the given condition, and hence become more suitable for extracting specific relationships for the condition. Extensive experiments and analyses on three different tasks that cover ten datasets across five different languages demonstrate the generality and effectiveness of TAGPRIME.

heading: 1 Introduction
text: There are many tasks in natural language processing (NLP) that require extracting relational structures from texts. For example, the event argument extraction task aims to identify event arguments and their corresponding roles for a given event trigger (Huang et al., 2022; Wang et al., 2019). In entity relation extraction, the model identifies the tail-entities and head-entities that forms specific relations (Wei et al., 2020; Yu et al., 2020). In taskoriented semantic parsing, the model predicts the slots and their semantic roles for a given intent in an
∗The authors contribute equally.
utterance (Tür et al., 2010; Li et al., 2021). These tasks are beneficial to a wide range of applications, such as dialog systems (Liu et al., 2018), question answering (Yasunaga et al., 2021), and narrative generation (Chen et al., 2019a). Prior works usually design models to specifically address each of the tasks (Sun et al., 2019; Miwa and Bansal, 2016; Han et al., 2019; Fu et al., 2019; Zhang et al., 2018). However, less attention is paid to the commonality among these tasks and having a unified framework to deal with them and provide a strong baseline for every task.
In this work, we take a unified view of these NLP tasks. We call them relational structure extraction (RSE) tasks and formulate them as a unified task that identifies arguments to a given condition and classifies their relationships. The condition could be a textual span, such as an event trigger for event argument extraction, or a concept, such as an intent for task-oriented semantic parsing.
We present TAGPRIME, a simple, unified, and strong model, which follows a sequence tagging paradigm with a priming technique, which is proposed by Fincke et al. (2022). TAGPRIME inherits the strength of sequence tagging models to unifiedly address RSE by converting the relational structure into a sequence of predictions by sequentially labeling tokens in the input passage. TAGPRIME further improves this framework’s performance by better incorporating information about the given condition via priming. Traditional sequence tagging models usually leverage learnable feature embeddings to incorporate information about the given condition before the tags are assigned, as illustrated in Figure 1(a). With the priming mechanism, TAGPRIME augments the input text with condition-specific contexts, as illustrated in Figure 1(b) & (c). The main merit of the
12917
priming technique comes from the nature of the self-attention mechanism in pre-trained language models. Augmenting input text with conditionspecific contexts makes the sentence representations condition-specific directly. Thus, it unlocks the capability of sequence tagging methods for relational structure extraction better than the commonly used feature embedding approach, as shown in Section 5.
Our contributions can be summarized as follows. (1) We take a unified view of NLP tasks that requires extracting relational structures, including end-to-end event extraction, end-to-end relation extraction, and task-oriented semantic parsing. Then, we present TAGPRIME, a unified sequence tagging model with priming that can serve as a strong baseline to various relational structure extraction problems. (2) Thorough experiments on three different tasks show that TAGPRIME achieves competitive performance than the current state-of-the-art on ten datasets in five different languages. (3) We propose
a novel efficient approximation to speed up TAGPRIME during inference time without sacrificing too much performance.
Our code will be publicly accessible at https: //github.com/PlusLabNLP/TagPrime.

heading: 2 Related Work
text: Many natural language processing applications require extracting relational structures, including event extraction, relation extraction, coreference resolution, etc. The prevalence of these applications makes us hard to exhaustively list them in this short summary, hence, we mainly focus on related works for the applications we experiment on.
Event extraction. Early works in event extraction mostly consider a pipelined approach (Nguyen and Grishman, 2015; Wang et al., 2019; Yang et al., 2019) to deal with event extraction. Some followup works argue that pipelined design leads to error propagation issues and hence propose end-to-end approaches to better capture dependencies between
each prediction (Lin et al., 2020; Li et al., 2013; Nguyen et al., 2016; Hsu et al., 2022b; Lu et al., 2021; Huang and Peng, 2021). However, recently, some empirical studies (Hsu et al., 2022b; Zhong and Chen, 2021; Fincke et al., 2022) also show that when an abundant amount of data is used to learn representations for each pipelined task, it is hard to conclude that joint learning approaches always provide a stronger result. This aligns with our discovery in experiments — even though we apply a pipelined approach with a simple sequence tagging framework on event extraction, with the help of priming to learn more condition-aware contextualized representation, we can still achieve very strong performance on multiple datasets.
Relation extraction. End-to-end relation extraction can usually be solved using two categories of approaches. The first one is to directly perform joint inference on named entities and their relation(s) (Zheng et al., 2017; Wang and Lu, 2020; Katiyar and Cardie, 2017; Sun et al., 2019; Miwa and Bansal, 2016; Fu et al., 2019). The second category is to perform a pipeline that first extracts named entities, and then performs relation classification (Wu and He, 2019; Hsu et al., 2022a; Lyu and Chen, 2021; Peng et al., 2020; Zhou and Chen, 2021a; Lu et al., 2022), which assumes that both the head-entity and tail-entity are given. Yet, in our unified formulation for relational structure extraction tasks, we extract tail-entities and their corresponding relation types for a given head-entity, which is more similar to a less frequently studied framework called cascading approaches (Wei et al., 2020; Yu et al., 2020). Despite being a less popular formulation to deal with end-to-end relation extraction, TAGPRIME presents a strong performance compared to prior studies, showcasing the practicality and effectiveness of our unified formulation.
Task-oriented semantic parsing. Task-oriented semantic parsing, which focuses on intent classification and slot filling, has a long history of development (Tür et al., 2010; Gupta et al., 2018; Li et al., 2021; Zhang et al., 2018; Louvan and Magnini, 2020). Recently, some more advanced neural network-based approaches have been proposed, such as MLP-mixer (Fusco et al., 2022) or sequence-to-sequence formulation (Desai et al., 2021). Among them, JointBERT (Chen et al., 2019b), a sequence-tagging-based model that is trained to jointly predict intent and extract slots,
serves as a widely-used baseline due to its simplicity. Our approach benefits from the same simplicity as JointBERT and can further improve its performance.

heading: 3 Method
text: We first introduce our view to unify RSE problems and then discuss how TAGPRIME approaches this problem under a unified framework of sequence tagging model with priming.

heading: 3.1 A Unified Formulation of RSE
text: Given an input text x = [x1, x2, ..., xn] and a condition c, The RSE tasks identify a list of spans sc = [sc1, s c 2, ..., s c l ] and their corresponding relationships or attributes rc = [rc1, r c 2, ..., r c l ] towards the condition c, where rci ∈ A and A is the set of all possible relationships or attributes. Many NLP tasks can be formulated as an RSE task. We showcase how this formulation can be applied to event extraction, entity relation extraction, and taskoriented semantic parsing below.
End-to-end event extraction. End-to-end event extraction aims to extract events from given texts (Ma et al., 2020; Hsu et al., 2022b; Yang et al., 2019). An event contains a trigger, which is the textual span that best represents the occurrence of an event, and several arguments, which are the participants involved in the event with different argument roles. We consider a pipeline solution — after the event triggers are identified, an argument extraction model extracts the event arguments and their corresponding roles for each given event trigger. Under the RSE formulation, the condition c is the given event trigger, and the target spans sc and the relationships rc are the arguments and their argument roles, respectively.
End-to-end relation extraction. Relation extraction identifies entities and their relations from texts, and it is usually solved by pipeline approaches — first extracting named entities and then predicting relations for each entity-pair (Wu and He, 2019; Zhong and Chen, 2021). Under the new formulation, an RSE model is used to predict tail-entities and the relations for each extracted named entity that serves as the head-entity. For example, in Figure 1(b), we extract the tail-entities (“Iraqi” and “base”) and their relation (“Part-Whole” and “ART”) for the head-entity, “military”. In this way,
each given head-entity is the condition c, and the extracted tail-entities are sc, with relations, rc.
Task-oriented semantic parsing. Task-oriented semantic parsing aims to classify the intent and parse the semantic slots in an utterance (to a taskoriented dialog system) (Li et al., 2021; Gupta et al., 2018). To fit into our formulation, we first predict the intent and then use a relational structure extraction model to predict the slots (sc) as well as their semantic roles (rc) for the given intent (c).

heading: 3.2 Sequence Tagging Model for RSE
text: We hereby introduce the typical way of applying a sequence tagging model to unifiedly solve relational structure extraction. The goal of our sequence tagging model for relational structure extraction is to predict the BIO-tag sequence y = [y1, y2, ..., yn], where each yi is the corresponding tag for each token xi in the input text. The BIOtag sequence can then be decoded to represent the extracted spans sc (and their relationships rc).
Specifically, given an input text, we obtain the contextualized representation zi for each token xi by passing the passage to a pre-trained language model.1 To embed the information of the condition c, one commonly-used technique is to add conditional features to zi (Ma et al., 2020; Wei et al., 2020; Yang et al., 2019; Yu et al., 2020), as shown in Figure 1(a). For example, in Ma et al. (2020), they use a token embedding of the given event trigger word and a learnable event type feature as the conditional features for the task of event argument extraction. In such case, the feature of c will contain the contextualized word representation zj , if xj is the token that represents the given condition, i.e., event trigger. In our experimental setup, if the given condition can be represented as an input span, we will include the span embeddings as the conditional features together with the type embeddings, such as the cases for event extraction and relation extraction. If the condition is only a concept, such as the task-oriented semantic parsing case, the conditional features will only contain type embeddings. Augmented with these conditional features, the final representation for token xi is further fed into multi-layer perceptrons and a conditional random field (CRF) layer (Lafferty et al., 2001) to predict the BIO-tag sequence y, as
1If a token xi is split into multiple word pieces, we use the average embeddings of all its word pieces to be zi, following the practice of Lin et al. (2020).
illustrated in Figure 1(a).

heading: 3.3 TAGPRIME
text: TAGPRIME follows the sequence tagging paradigm but utilizes the priming technique for better leverage information about the input condition.
Condition Priming. Motivated by previous work (Fincke et al., 2022), we consider priming to inject the information of the condition c to further improve the sequence tagging model. The priming mechanism informs the model of the conditional information by directly appending conditional information to the input text. However, unlike Fincke et al. (2022) that uses an integer string to represent features in a categorical style, we use a naturallanguage-styled indicator to better exploit the semantics of the condition. The indicators can be obtained by verbalizing the conditional information.
Take Figure 1(b) as an example, when extracting the tail-entities and the relationships for the “military” head-entity (condition c), we first verbalize the entity type of “military”, i.e., from “Org” to “Organization”. Then, the string “military” and “Organization” are appended to the input text, which serves as the information about the condition c.
The priming technique leverages the selfattention mechanism in pre-trained language models and makes the token representation zi conditionaware. Hence, the representation of every zi is more task-specific than the one in the model described in Section 3.2. More precisely, for tagging models without priming, the representation zi usually captures more general information that focuses on the context of input text. For models with priming, the representation zi is affected by the additional verbalized words when computing attention. Hence, zi becomes more task-specific and more suitable for addressing the task (Zheng and Lapata, 2022; Zhong and Chen, 2021). Additionally, the priming method can be easily combined with conditional features described in Section 3.2. More discussion on this will be shown in Section 5.
Relationship Priming. The same idea of condition priming can also be extended to relationship. Specifically, we decompose a relational structure extraction task into several extraction subtasks, each of them only focusing on one single relationship r (r ∈ A). Similar to the condition priming, we verbalize the relationship information and append related strings to the input text as well. Therefore, the representation zi is aware of the relation-
ship r and specific for predicting spans with relationship r to the condition c.
For example, in Figure 1(c), for the given relationship “Part-Whole”, we first verbalized it into
“is part of”. Then, the string “is part of” is appended to the input text together with the condition priming strings. The BIO-tag sequence can be decoded into those tail-entities sc that form “Part-Whole” relationship(s) with the given head-entity “military”.
Discussion. A similar idea of appending tokens in the pre-trained language model’s input to affect the output text representation has also been leveraged in Zhou and Chen (2021b); Zhong and Chen (2021). Yet, different from their works that only focus on relation classification and apply instancespecific information, our TAGPRIME with relationship priming method focuses on using task-specific information, because we decompose relational extraction into sub-tasks. We want that different taskspecific representation can be learned for different sub-tasks, hence proposing relationship priming.
An underlying advantage of TAGPRIME with relationship priming is its ability to handle cases containing multi-relationships. After we decompose a relational structure extraction task into several extraction subtasks, we do not perform any filtering to address conflict relationship predictions between the same condition and extracted span. This is to enlarge our model’s generality to different scenarios.

heading: 4 Experiments
text: To study the effectiveness of TAGPRIME, we consider three NLP tasks: (1) end-to-end event extraction, (2) end-to-end relation extraction, and (3) taskoriented semantic parsing. All the results are the average of five runs with different random seeds.

heading: 4.1 End-to-End Event Extraction
text: Datasets. We consider the two most widely-used event extraction datasets, ACE-2005 (Doddington et al., 2004) and ERE (Song et al., 2015). For ACE-2005 (ACE05-E), we experiment on the English and Chinese portions and keep 33 event types and 22 roles, as suggested in previous works (Wadden et al., 2019; Hsu et al., 2022b). For ERE, we consider the English and Spanish annotations and follow the preprocessing of Lin et al. (2020) to keep 38 event types and 21 roles.
Baselines. We consider the following end-to-end event extraction models, including DyGIE++ (Wad-
den et al., 2019), TANL (Paolini et al., 2021), Text2Event (Lu et al., 2021), OneIE (Lin et al., 2020), and DEGREE (Hsu et al., 2022b). Since TAGPRIME requires trigger predictions, we simply take the trigger predictions made by a simple sequence tagging model trained with multi-tasking on trigger detection and named entity recognition.
For TAGPRIME, DyGIE++, and OneIE, we consider BERT-large (Devlin et al., 2019) for ACE05-E (en) and ERE (en), and consider XLM-RoBERTalarge (Conneau et al., 2020) for ACE05-E (zh) and ERE (es). For generation-based models, we consider BART-large (Lewis et al., 2020) for DEGREE, T5-base (Raffel et al., 2020) for TANL, and T5large (Raffel et al., 2020) for Text2Event, as suggested by their original papers.
Implementation details. The followings are the training details for all baselines:
• DyGIE++ (Wadden et al., 2019): we use the released training script2 with the default parameters.
• TANL (Paolini et al., 2021): we report the numbers from the original paper.
• Text2Event (Lu et al., 2021): we report the numbers from the original paper.
• OneIE (Lin et al., 2020): we use the released training script3 with the default parameters.
• DEGREE (Hsu et al., 2022b): we report the numbers from the original paper.
• TAGPRIME (ours): We fine-tune pre-trained language models with the dropout rate being 0.2. We use AdamW optimizer. For parameters that are not pre-trained we set the learning rate to 10−3 and the weight decay to 10−3. For parameters that are not pre-trained we set the learning rate to 10−5 and the weight decay to 10−5. We consider the linear scheduler with a warm-up, where the warm-up epoch is 5. The number of epochs is 90. The training batch size is set to 6. For conditional token features and learnable features, the dimension is set to 100. It takes around 6 hours to train with a NVIDIA RTX A6000 with 48GB memory.
Evaluation metrics. Following previous works (Wadden et al., 2019; Lin et al., 2020), we measure the correctness of arguments based on whether the offsets of the argument span match or not. We
2https://github.com/dwadden/dygiepp 3http://blender.cs.illinois.edu/
software/oneie/
consider argument identification F1-score (Arg-I), which cares about only the offset correctness, and argument classification F1-score (Arg-C), which cares about both offsets and the role types. We also report trigger classification F1-score (Tri-C), although it is not our main focus as the triggers are provided via other models and we just use their predictions to simulate the end-to-end scenarios.
Results. Table 1 shows the results of end-to-end event extraction on various datasets and languages. Although simple, TAGPRIME surprisingly has decent performance and achieves better results than the state-of-the-art models in terms of argument F1scores. We attribute the good performance to the design of priming, which leverages the semantics of the condition and makes the representations more task-specific. It is worth noting that considering relationship priming further improves the results, which again shows the importance of task-specific representations.

heading: 4.2 End-to-End Relation Extraction
text: Datasets. We consider two popular end-toend relation extraction datasets, ACE04 and ACE05 (Doddington et al., 2004), denoted as ACE04-R and ACE05-R. Both datasets consider 7 named entity types and 6 different relations. We follow the same procedure in Zhong and Chen (2021) to preprocess the data and split the datasets. We refer readers to their papers for more details about the datasets.
Baselines. We compare to the following end-toend relation extraction models: Table-Sequence (Wang and Lu, 2020), PFN (Yan et al., 2021), and Cascade-SRN (both late fusion and early fusion) (Wang et al., 2022). Additionally, we consider PURE (Zhong and Chen, 2021), which also takes a pipelined approach to solve end-to-end relation
extraction. To fairly compare with prior works, we use PURE’s named entity predictions on the test set for TAGPRIME to perform relational structure extraction.4 In order to be consistent with our other tasks, we adopt the single sentence setting (Zhong and Chen, 2021) for our model. However, we also list baselines with cross-sentence settings, such as PURE’s and UniRE (Wang et al., 2021)’s results with cross-sentence context as input. All the models use ALBERT-xxlarge-v1 (Lan et al., 2020) as the pre-trained language models.
Implementation details. The followings are the training details for all baselines:
• Table-Sequence (Wang and Lu, 2020): we report the numbers from the original paper.
• Cascade-SRN (Wang et al., 2022): we report the numbers from the original paper.
• PURE (Zhong and Chen, 2021): we report the numbers from the original paper.
• PFN (Yan et al., 2021): we report the numbers from the original paper.
• UniRE (Wang et al., 2021): we report the numbers from the original paper.
• TAGPRIME (ours): We fine-tune pre-trained language models with the dropout rate being 0.2. We use AdamW optimizer. For parameters that are not pre-trained we set the learning rate to 10−3 and the weight decay to 10−3. For parameters that are not pre-trained we set the learning rate to 2× 10−5 and the weight decay to 10−5. We consider the linear scheduler with a warm-up, where the warm-up epoch is 5. The number of epochs is 30. The training batch size is set to 32. For conditional token features and learnable features, the dimension is set to 100.
4We get PURE’s named entity recognition predictions by retraining PURE’s named entity recognition model.
It takes around 20 hours to train with a NVIDIA RTX A6000 with 48GB memory.
Evaluation metrics. We follow the standard evaluation setting with prior works (Bekoulis et al., 2018; Zhong and Chen, 2021) and use micro F1score as the evaluation metric. For named entity recognition, a predicted entity is considered as a correct prediction if its span and the entity type are both correct. We denote the score as “Ent” and report the scores even though it is not our main focus for evaluation. For relation extraction, two evaluation metrics are considered: (1) Rel: a predicted relation is considered as correct when the boundaries of head-entity span and tail-entity span are correct and the predicted relation type is correct; (2) Rel+: a stricter evaluation of Rel, where they additionally required that the entity types of head-entity span and tail-entity must also be correct.
Results. The results of end-to-end relation extraction are presented in Table 2. From the table, we observe that TAGPRIME has the best performance on ACE05-R and outperforms most baselines on ACE04-R. This shows the effectiveness of TAGPRIME. Similar to the results of event extraction, considering relationship priming makes the representations more relationship-aware and leads to performance improvement.

heading: 4.3 Task-Oriented Semantic Parsing
text: Datasets. We choose MTOP (Li et al., 2021), a multilingual dataset on semantic parsing for taskoriented dialog systems. We specifically consider data in English (en), Spanish (es), French (fr), and German (de).
Baselines. We consider JointBERT (Chen et al., 2019b), the commonly used baseline for task-
oriented semantic parsing. We directly use the predicted intents by JointBERT as the condition of TAGPRIME for a fair comparison. Both TAGPRIME and JointBERT are trained with XLM-RoBERTalarge (Conneau et al., 2020). Unlike event extraction and relation extraction, the condition of taskoriented semantics parsing (intent) does not include the word span, therefore, only a type feature embedding is contained in the conditional features for TAGPRIME in this experiment.
Implementation details. The followings are the training details for all baselines:
• JointBERT (Chen et al., 2019b): we use the training script5 with the default parameters.
• TAGPRIME (ours): We fine-tune pre-trained language models with the dropout rate being 0.2. We use AdamW optimizer. For parameters that are not pre-trained we set the learning rate to 10−3 and the weight decay to 10−3. For parameters that are not pre-trained we set the learning rate to 10−5 and the weight decay to 10−5. We consider the linear scheduler with warm-up, where the warm-up epoch is 5. The number of epochs is 90. The training batch size is set to 6. For conditional token features and learnable features, the dimension is set to 100. It takes around 4 hours to train with a NVIDIA RTX A6000 with 48GB memory.
Evaluation metrics. We following MTOP (Li et al., 2021) to consider slot identification (Slot-I) and slot classification (Slot-C) F1-scores. Even though we focus on the performance of slot filling, we also report the intent classification accuracy.
Results. As demonstrated in Table 3, TAGPRIME 5https://github.com/monologg/JointBERT
achieves a better performance than the baselines. Again, considering relationship priming leads to further improvement. It is worth noting that TAGPRIME is effective for different languages, which shows the generality of TAGPRIME.

heading: 4.4 Summary
text: We show the superiority of TAGPRIME on three different tasks (including ten different datasets across five different languages). Although being a unified and simple model, the results suggest that TAGPRIME can achieve competitive results for tasks requiring extracting relational structures.

heading: 5 Analysis
text: In this section, we study two questions: (1) What is the effectiveness of priming techniques compared to learnable features? (2) Relationship priming boosts the performance of TAGPRIME, but the task decomposition could slightly slow down the inference speed. Can we mitigate this issue?
To answer the first question, we conduct ablation experiments on sequence tagging models using different combinations of learnable features or/and adding information through the priming technique (Section 5.1). For the second question, we propose a simple modification to TAGPRIME so that we can flexibly control the number of layers to fuse priming information to contextualized representations.
The modified TAGPRIME can serve as an efficient approximation of TAGPRIME (Section 5.2).

heading: 5.1 Ablation Study
text: We focus on the setting where we alter the choices on how to include the type information of the condition c and the relationship information r. Table 4 demonstrates our experimental results.
Comparing the first four cases in Table 4, we observe that the addition of type features is useful in general, and using the priming technique is a more effective way to incorporate conditional information. For models in case 5 to case 8, the relationship decomposition formulation described in Section 3.3 is applied. Comparing case 2 to case 5, we can see that simply applying the relationship decomposition formulation for solving relational structure extraction does not lead to improvements if the way to embed the relationship r is only through learnable features. However, comparing case 3 to case 6 and case 4 to case 7, we show that the relationship priming approach makes the representation zi well capture the attribute of the queried relationship, thus, better exploiting the advantage of the relationship decomposition formulation and gaining improvements. Note that we conducted preliminary experiments that use pretrained language models’ representations of the same verbalized token to be the initialization of the learnable type feature embedding, but the method
shows similar results with the random initialization, hence, we stick to random initialization on the learnable type features.

heading: 5.2 Efficient approximation of TAGPRIME
text: To make TAGPRIME to inference faster, we perform two modifications to TAGPRIME: (1) We first separate the pre-trained language model, which contains L layers, into two halves — one with the first k layers, the other one is the remaining layers. (2) We copy the first half of the language model to another module. When an input passage is fed into the model. We use the original first half to encode the input text as well as the verbalized condition, and we use the copied first half to encode the verbalized relation. Finally, the encoded representations will be fed into the second half layers, as illustrated in Figure 2. The value of k is adjustable, where when k = 0, it represents the TAGPRIME with condition and relationship priming, and when k = L, it is TAGPRIME with condition priming.
Since the encoding stage of the input text and the verbalized relationship is separated, we can accelerate the inference time of our modified TAGPRIME through parallel encoding. More precisely, our modified TAGPRIME can aggregate instances that share the same passage and verbalized condition. For those instances, TAGPRIME only needs to perform the encoding once on their input passage part,6 and paired with several separated embedded verbalized relationships, which could be parallelly encoded together.
We conduct experiments on the ACE05-E (en)
6The string of verbalized relationship is usually much shorter than the input passage, hence, in most cases, the major part of the input for an instance is the input text and which requires more computations.
dataset to test our modification. In order to better analyze the results and isolate the influence from the pipelined errors, we report the results on the event argument extraction when gold event triggers are given. The experimental results are shown in Figure 3. First, we investigate the performance influence of our modification. We find that when k ≤ 10, the performance of our modified TAGPRIME is strong in general and is comparable with TAGPRIME with the condition and relationship priming. To compare the efficiency of the model, we benchmark the inference time by performing inference on the whole testing dataset fifty times and calculate the average speed, which is measured by checking how many instances can be processed per second. The red line in Figure 3 shows the results. We observe that for our modified TAGPRIME with k = 10, its inference speed is around 30% faster than the TAGPRIME with the condition and relationship priming, but they perform similarly.

heading: 6 Conclusion
text: In this work, we take a unified view of tasks requiring extracting relational structures and present TAGPRIME, a simple, unified, effective, and general sequence tagging model. The key idea is applying priming, a small trick to make the representations task-specific by appending condition-related and relationship-related strings to the input text. Our experimental results demonstrate that TAGPRIME is general to different tasks in various languages and can serve as a strong baseline for future research on relational structure extraction.

heading: Acknowledgments
text: We thank anonymous reviewers for their helpful feedback. We thank the UCLA PLUSLab and UCLA-NLP group members for the valuable discussions and comments. This research was supported in part by AFOSR MURI via Grant #FA9550-22-1-0380, Defense Advanced Research Project Agency (DARPA) via Grant #HR00112290103/HR0011260656, the Intelligence Advanced Research Projects Activity (IARPA) via Contract No. 2019-19051600007, National Science Foundation (NSF) via Award No. 2200274, and a research award sponsored by CISCO.
Limitations
As we point out in Section 5, one of the limitations in TAGPRIME is the inference speed. When we perform TAGPRIME with condition and relationship priming, we requires more turns of sequence tagging processes than typical sequence tagging models. Observing this, we propose a simple way to mitigate such issue and increase the inference speed with only a small performance drop. Despite such effort, it is still slightly slower than the model requires only one pass of sequence labeling.
The other potential limitation of our method is that we assume the condition and relationship can be verbalized. However, in practice, there could be cases that the verbalization is hard to be done. Considering this, we do conduct preliminary experiments of applying TAGPRIME with special tokens priming rather than verbalized tokens. However, our preliminary results show that such method’s performance is less stable and weaker than we can achieve with TAGPRIME.
Ethics Considerations
TAGPRIME fine-tunes the pre-trained language models (Devlin et al., 2019; Lan et al., 2020). There have been works showing the potential bias in pretrained language models. Although with a low possibility, especially after our finetuning, it is possible for our model to make counterfactual, and biased predictions, which may cause ethical concerns. We suggest carefully examining those potential issues before deploying the model in any real-world applications.

heading: A Detailed Results
text: Table 5, 6, and 7 lists the detailed results (mean and standard deviation) of TAGPRIME.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Please see the "Limitations" section.

heading: 3 A2. Did you discuss any potential risks of your work?
text: Please see the "Ethics Considerations" section

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 4

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 4

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: Section 4

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Section 4
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? Not applicable. Left blank.
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 4
C 3 Did you run computational experiments? Section 4

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Section 4 and Appendix A
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Section 4 and Appendix A

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 4 and Appendix B

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 4 and Appendix A
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: Our experimental results demonstrate that TAGPRIME is general to different tasks in various languages and can serve as a strong baseline for future research on relational structure extraction