Coarse-to-Fine Dual Encoders are Better Frame Identification Learners (2023)

Abstract: Frame identification aims to find semantic frames associated with target words in a sentence. Recent researches measure the similarity or matching score between targets and candidate frames by modeling frame definitions. However, they either lack sufficient representation learning of the definitions or face challenges in efficiently selecting the most suitable frame from over 1000 candidate frames. Moreover, commonly used lexicon filtering ( lf ) to obtain candidate frames for the target may ignore out-of-vocabulary targets and cause inadequate frame modeling. In this paper, we propose CoFFTEA, a \underline{Co} arse-to- \underline{F} ine \underline{F} rame and \underline{T} arget \underline{E} ncoders \underline{A} rchitecture. With contrastive learning and dual encoders, CoFFTEA efficiently and effectively models the alignment between frames and targets. By employing a coarse-to-fine curriculum learning procedure, CoFFTEA gradually learns to differentiate frames with varying degrees of similarity. Experimental results demonstrate that CoFFTEA outperforms previous models by 0.93 overall scores and 1.53 R@1 without lf . Further analysis suggests that CoFFTEA can better model the relationships between frame and frame, as well as target and target. The code for our approach is available at https://github.com/pkunlp-icler/COFFTEA.

## introduction frame identification @xcite aims to identify a semantic frame for the given target word in a sentence. the frames defined in framenet @xcite are associated with word meanings as encyclopedia knowledge @xcite , which represent different events, situations, relations and objects. identifying frames can contribute to extracting frame semantic structures from a sentence, and frame semantic structures can be utilized in downstream tasks, e.g., information extraction @xcite , machine reading comprehension @xcite . previous researches in frame identification selects an appropriate frame for a target by modeling the similarity or matching score between the target and frames. recent efforts are made to represent frames with definitions in framenet @xcite @xcite . however, limited by the large scale of frames, these methods either struggle to effectively model definitions due to the use of a frozen frame encoder or cannot efficiently select appropriate frames from all frames due to the use of a single encoder architecture. fig. 1 illustrates an example of frame identification. for the target word "get" in the sentence, we need to select its corresponding frame from over a thousand frames in framenet (all): board_vehicle (a traveller boards a vehicle). previous frame identification methods @xcite typically apply lexicon filtering (lf ) by selecting the frame from a set of candidate frames (candidates) for the verb "get" (e.g., arriving, getting, board_vehicle, etc.). lf effec-tively improves the accuracy of frame identification. however, the manually defined lexicons is not possible to cover all the corresponding targets for frame, which hinders the extension of frame semantics to out-of-vocabulary target words. furthermore, we observed that training w/o lf (without lf ) poses challenges in distinguishing finegrained differences among candidate frames. conversely, training w/ lf (with lf ) can lead to an increased distance between the gold frame and other candidates, making it difficult to distinguish the gold frame from irrelevant ones. consequently, in this paper, our aim is to enhance our model's performance in both the w/ lf and w/o lf scenarios. in this paper, we propose cofftea, a coarse-to-fine frame and target encoder architecture. by employing a dual encoder structure, we can effectively store the representations of all frames during inference, enabling us to efficiently compute the similarity between the target and all frames. furthermore, considering the constraints imposed by the number of frames, we incorporate contrastive objectives to enhance the trainability of the frame encoder, consequently improving our ability to model definitions more accurately. additionally, employing a distinct frame or target encoder allows us to map certain frames and targets to a vector space and utilize distance measures to depict their relationship. to address the trade-off between training w/ lf and w/o lf, we propose a two-stage learning procedure, referred to as coarse-to-fine learning. in the first stage, we employ in-batch learning, using the gold frames of other instances within a batch as hard negative examples. this step helps the model learn to discern frames with distinct semantics. in the second stage, we adopt in-candidate learning, where we utilize other frames from the candidate set as negative samples. this enables the model to further refine its ability to recognize fine-grained semantic differences. with coarse-to-fine learning, cofftea achieves a comprehensive performance in both scenarios of w/ lf and w/o lf. experiments show that cofftea, compared to previous methods on two framenet datasets, achieves competitive accuracy w/ lf and outperforms them by up to 1.53 points in r@1 w/o lf and 0.93 overall scores considering both settings. ablations also verify the contributions of the dual encoder structure and the two-stage learning process. further analyses suggest that both the target encoder and frame encoder can effectively capture the target-target, and frame-frame relationships. our contributions can be summarized as follow: ## task formulation for a sentence @xmath0, • • • , w n with a target span @xmath1, • • • , w te , frame identification is to select the most appropriate frame @xmath2, where f denotes all frames in framenet. as framenet @xcite provides each frame with an associated set of lexical units (lus), we can obtain the lu of target t by utilizing its lemma and part-of-speech (pos) in the form lemma.pos (e.g., get.v) and apply lf to retrieve the lexicon filter candidate frames f t of t. then we focus on selecting @xmath3. ## methodology to model the alignment between frames and targets efficiently and effectively, in this paper, we propose cofftea, a dual encoder architecture with two-stage contrastive objectives. we compare cofftea with other baselines modeling frame definitions ( § 3.1). specifically, to achieve better represent learning of definitions, the frame encoder of cofftea is learnable during training. moreover, in order to make cofftea gradually learn to differentiate frames with varying degrees of similarity. we introduce two-stage learning in §3.2. ## experiment our experiments mainly cover three aspects: ## related work frame identification prior research in frame identification has primarily focused on modeling the similarity between targets and frames or assessing the matching score between them. in terms of modeling similarity, hartmann et al. (2017); peng et al. (2018); zheng et al. (2022) leverage target encoder with a trainable frame representation table, while ignoring the valuable contextual frame information provided by framenet. to remedy this deficiency, @xcite ; @xcite both propose to incorporate frame definitions into frame encoder. despite these efforts, the number of frames and the optimizing objective make it infeasible to update frame representation during training, leading to a compromise of freezing it. consequently, they all trap in the dilemma to obtain optimal frame representation. as for assessing the matching score, @xcite propose a fused encoder to evaluate semantic coherence between sentence and frame definition. nevertheless, it just fails to evaluate the performance of w/o lf scenario. in addition, the aforementioned studies tend to adopt a one-sided relationship between targets and frames. they only train model on either w/ lf or w/o lf, which can not catch an overall horizon in distinguishing frames. moreover, only @xcite release performance under w/o lf scenario, which is essential for evaluating the full capabilities of the model. some other work like chanin (2023) treat frame semantic parsing as a sequence-to-sequence text generation task, but its still trap in training under w/ lf and can not catch an optimal relationships. metric learning metric learning is a fundamental approach in machine learning that seeks to learn a distance metric or similarity measure that accurately captures the inherent relationships between data points. one particular technique that has gained prominence in this area is contrastive learning, which has been extensively applied to a variety of tasks such as text classification @xcite , image classification @xcite . the optimized feature space retrieved from contrastive learning objective can benefit on learning better representations of data and capturing inner relationship. for frame identification, the primary goal is to obtain a well-aligned target-to-frame space. however, existing researches only focus on using cross entropy to directly modeling correlation to all frames, which presents challenges in capturing complex relationships. alternatively, contrastive learning can be leveraged to optimize and incrementally construct the feature space of frames and targets. ## conclusion we propose cofftea, and its dual encoder can effectively model the definitions with learnable frame encoder and efficiently calculate the similarity scores between a target and all frames. we also conclude that there exists a trade-off between training w/ lf and w/o lf, and achieves a comprehensive performance with a two stage learning process. detailed analysis of the experiments further demonstrates that our two encoders are not limited to their alignment but can also discover other connections between frame and frame, as well as target and target.