Coarse-to-Fine Dual Encoders are Better Frame Identification Learners
Authors: Kaikai An
Ce Zheng
Bofei Gao
Haozhe Zhao
Baobao Chang
Venue: EMNLP 2023 Findings (2023)
Keywords: Frame Identification
Frame Semantics
Contrastive Learning
Metric Learing

Abstract: Frame identification aims to find semantic frames associated with target words in a sentence. Recent researches measure the similarity or matching score between targets and candidate frames by modeling frame definitions. However, they either lack sufficient representation learning of the definitions or face challenges in efficiently selecting the most suitable frame from over 1000 candidate frames. Moreover, commonly used lexicon filtering ($lf$) to obtain candidate frames for the target may ignore out-of-vocabulary targets and cause inadequate frame modeling. In this paper, we propose CoFFTEA, a $\underline{Co}$arse-to-$\underline{F}$ine $\underline{F}$rame and $\underline{T}$arget $\underline{E}$ncoders $\underline{A}$rchitecture. With contrastive learning and dual encoders, CoFFTEA efficiently and effectively models the alignment between frames and targets. By employing a coarse-to-fine curriculum learning procedure, CoFFTEA gradually learns to differentiate frames with varying degrees of similarity. Experimental results demonstrate that CoFFTEA outperforms previous models by 0.93 overall scores and 1.53 R@1 without $lf$. Further analysis suggests that CoFFTEA can better model the relationships between frame and frame, as well as target and target. The code for our approach is available at https://github.com/pkunlp-icler/COFFTEA.

Coarse-to-Fine Dual Encoders are Better Frame Identification Learners
Kaikai 2 Ce Zheng", Bofei Gao', Haozhe Zhao' Baobao Chang"!
National Key Laboratory for Multimedia Information Processing,
School of Computer Science, Peking University
School of Software & Microelcctronics, Peking University 2zs1x, 2 gaobofei, hanszhao pku. edu. cn
{ankaikai, zce11 seeaakkai,zee11111 chbbêpku. edu. cn
Abstract
Candidate Frames (10) Come down with Ariving Board vehicle Getting
All Frames (1221)
Getting
Frame identification aims to find semantic frames associated with target words in a sen- tence. Recent researches measure the similar- ity or matching score between targets and can- didate frames by modeling frame definitions. However, they either lack sufficient represen- tation learning of the definitions or face chal- lenges in efficiently selecting the most suitable frame from over 1000 candidate frames. More- over, commonly used lexicon filtrrin (If) to obtain candidate frames for the target may ig- nore out-of- -oolo-oblcabblryy targets and cause in- adequate frame modeling. In this paper, we propose COFFTEA, Coarse-to-Fine Frame and Target Encoders Architecture. With con- trastive learning and dual encoders, COFFTEA efficiently and effectively models the align- ment between frames and targets. By employ- ing coarse-to-fine curriculum learning proce- dure, COFFTEA gradually learns to differentiate frames with varying degrees of similarity. Ex- perimental results demonstrate that COFFTEA outperforms previous models by 0.93 overall scores and 1.53 R@ without Further anal- ysis suggests that COFFTEA can better model the relationships between frame and frame, as well target and target. The code for our ap- proach is available at https://g/ thub. com/ pkunlp-icler/COFFTE..
Trunsition state Board vehicle Surpassing
m tf
w/
Can he just get plane and fly to Paris? get.v Board vehicle
An example Frame Identification. The target word get the sentence is associated with frame Board vehicle. Lexicon filtering (i)) considers frames that can be evoked by get.v as candidates.
Figure 1:
frame semantic structures can be utilized in down- stream tasks, e.g., Information Extraction (Si and Roberts, 2018), Machine Reading Comprehension (Guo et al., 2020).
Previous researches in frame identification se- lects an appropriate frame for a target by model- ing the similarity or matching score between the target and frames. Recent efforts are made to rep- resent frames with definitions in FrameNet (Jiang and Riloff, 2021; Su et al., 2021; Tamburini, 2022; Zheng et al., 2023). However, limited by the large scale of frames, these methods either struggle effectively model definitions due to the use of a frozen frame encoder or cannot efficiently select appropriate frames from all frames due to the use of a single encoder architecture.
1 Introduction
Frame Identification (Gildea and Jurafsky, 2000; Baker et al., 2007) aims to identify semantic frame for the given target word sentence. The frames defined in FrameNet (Baker et al., 1998) are associated with word meanings as encyclope- dia knowledge (Fillmore et al., 1976), which rep- resent different events, situations, relations and ob- jects. Identifying frames can contribute to extract- ing frame semantic structures from sentence, and
Fig. illustrates an example of frame identifica- tion. For the target word "get" in the sentence, we need to select its corresponding frame from over a thousand frames in FrameNet (All): Board_vehicle (A Traveller boards Vehicle). Previous frame identification methods (Peng et al., 2018; Jiang and Riloff, 2021) typically apply lexicon filtering (If) by selecting the frame from a set of candi- date frames (Candidates) for the verb "get" (c.g.. Arriving, Getting, foard_vehicle, etc.). If effec-
"Equal contribution
Corresponding author

---

tively improves the accuracy of frame identifica- tion. However, the manually defined lexicons is not possible to cover all the corresponding targets for frame, which hinders the extension of frame semantics to out-of-vocabulary target words.
encoder and frame encoder can effectively capture the target-target, and frame-frame relationships. Our contributions can be summarized as follow:
We propose COFFTEA, a dual encoder ar- chitecture with coarse-to-fine contrastive ob- jectives. COFFTEA can effectively model frame definitions with learnable frame en- coders and efficiently select the corresponding frame from over 1,000 frames.
Furthermore, we observed that training w/o If (without (f) poses challenges in distinguishing fine- grained differences among candidate frames. Con- versely, training w/ If (with If) can lead to in- creased distance between the gold frame and other candidates, making it difficult to distinguish the gold frame from irrelevant ones. Consequently, in this paper, our aim is to enhance our model's performance in both the w/ If and w/o If scenarios.
2.
We conclude the trade-off between w/ If and w/o lf. and emphasize the performance w/o If in the evaluation of frame identification. With two-stage learning, COFFTEA achieves an overall score improvement of 0.93 on frame identification.
In this paper, we propose COFFTEA, a Coarse-to- Fine Frame and Target Encoder Architecture. By employing a dual encoder structure, we can effec- tively store the representations of all frames during inference, enabling similarity between the target and all frames. Fur- thermore, considering the constraints imposed by the number of frames, we incorporate contrastive objectives to enhance the trainability the frame encoder, consequently improving our ability to model definitions more accurately. Additionally. employing distinct frame or target encoder allows map certain frames and targets to vector space and utilize distance measures depict their relationship.
Further in-depth analysis demonstrates that our frame and target encoders can better model the alignment between frame and tar- get, as well as the relationships between target and target, frame and frame.
efficiently compute the
2 Task Formulation
For sentence S = w.. wn, with a target span W/g. .... Wt, frame identification is to select the most appropriate frame f E F for t, where denotes all frames in FrameNet.
As FrameNet (Baker et al., 2007) provides each frame with an associated set of lexical units (LUs), we can obtain the LU of target by utilizing its lemma and part-of-speech (POS) in the form lemma.POS (e.g.. get.v) and apply If to retrieve the lexicon filter candidate frames F, of t. Then we focus on selecting Ft.
To address the trade-off between training w/ If and w/o If. we propose two-stage learning proce- dure, referred to as coarse-to-le learning. In the first stage, we employ in-batch learning, using the gold frames of other instances within a batch as hard negative examples. This step helps the model learn to discern frames with distinct semantics. In the second stage, adopt in-candidate learning, where we utilize other frames from the candidate set as negative samples. This enables the model to further refine ability to recognize fine-grained semantic differences. With coarse-to-fing learning, COFFTEA achieves a comprehensive performance in both scenarios of w/ If and w/o If.
Methodology
To model the alignment between frames and tar- gets efficiently and effectively, in this paper, we propose COFFTEA, a dual encoder architecture with two-stage contrastive objectives. We compare COFFTEA with other baselines modeling frame def- initions ($ 3.1). Specifically, to achieve better rep- resent learning of definitions, the frame encoder of COIFTEA is learnable during training. Morcover. in order make COFFTEA gradually learn dif- ferentiate frames with varying degrees of similarity. We introduce two-stage learning in $3.2.
Experiments show that COFFTEA, compared to previous methods on two FrameNet datasets, achieves competitive accuracy w/ If and outper- forms them by up 1.53 points in R @ w/o If and 0.93 overall scores considering both settings. Ablations also verify the contributions of the dual encoder structure and the two-stage learning pro- cess. Further analyses suggest that both the target
3.1 Model Architecture
In Fig. 2, we compare the dual- encoder architecture of COFFTEA with three categories of previous meth-

---

Sentence:
Can just get on a plane fly to Paris?
Frame: Board vehicle
Definition: A Traveller boards Vehicle that they intend to use as a means of transportation....
co. f)
cos f)
50
O0 O
100 ROO
Target Encoder
Target Encoder
mittaiize
Frame Encoder *
Loolup Tuble
Ceneegaae Loobup Table
sentence
frame
sentence
frame def
Ptt, is matching)
(t,
00
@
000
Fused Encoder
Target Encoder
Frame Encoder
[CLS]
sentence frame def
sentence
frame def
(c)
(d)
Figure 2: Comparison between COFFTEA and other baselines. (a) and (b) both fine-tune Lookup Table containing embeddings of frames with corss entropy loss. (b) uses frozen frame encoder to initialize the table. (c) employs a single fused encoder to measure the probability sentence and definition matching. For efficiency purposes, COFFTEA (d) also adopts dual-encoder structure and leverages contrastive loss unfreeze the frame encoder.
ods in frame identification: (a) no definition mod- eling (Hartmann et al., 2017; Zheng et al., 2022), (b) target encoder and frozen frame encoder (Su et al., 2021; Tamburini, 2022), and (c) single fused encoder (Jiang and Riloff, 2021). Limited by the number of frames in FrameNet, the dual encoder architecture with cross entropy loss (b) have to fix the parameters of the frame encoder while the sin- gle encoder architecture (c) have to train and infer under the w/ If setting. Therefore, we still use two encoders to separately encode the sentence with tar- gets and frame defimition, and employ contrastive learning to make the frame encoder trainable.
t MaxPooling (he,, ht.))
(2)
With the target encoder, we capture the contextual information of the target sentence and the sufficient modeling of context can be used to find out-of- vocabulary targets. Our goal is to not only ensure close proximity between the representations of the target and the frame but also to ensure that the target representations in different sentences, which activate the same frame, exhibit high degree of similarity.
3.1.2 Frame Encoder
Taking sehicle as frame name and its cor- responding definition "A Traveller boards a Vehi- cle that they intend to use means of trans- portation either passenger as def. Similarly, frame encoder converts D frame name def, denoted as w/,..., w into rep- resentations .,,.... Nym We follow Reimers and Gurevych (2019) to regard meanpooling of all to- kens as frame representations f.
We follow Su (2021); Tamburini (2022) to use PLMs (Devlin et al., 2019) as target encoder ($3.1.1) and frame encoder 1.3...2).
(2021); Jiang and Riloff
driver"
3.1.1 Target Encoder
The target encoder converts input sentence with n tokens W1,, Wn, into contextualized represen- tations hi....,h, The contextualized representa- tion t of target span Wt,...., Wt, (1 < t, < te < n) is the maxpooling of hi....,,hu hic (Zheng et al.. 2023).
My.... Rinn Encoder,(w(, f MeanPooling(MM kmm) (4)
w'm)
(3)
Through the frame encoder, we encode frame definitions to represent frames. Similar to the target
h1,... hn Encodor,(w,, ww) (1)

---

4
gel 000
[000] f1 Board_vchick
Datasets
exemplar
train
dev
test
[F]
42
catch.v.000
000
f2
Come_ Come_down__ee with
FN 1.5
153946
16621
2284
4428
1019
fg
grill.v 808
0001 f3 Questioning
B
FN 1.7
192431
19391
2272
6714
1221
44
abseenee.n
[ / Presence
Table I: Statistics of FrameNet dataset
(a)
gel.v
000 f1 Board vchicle
000 f2 Come down with
the two-stage learning lies in the construction of negative pairs B and C.
0 fs Getting
c
000 f Arriving
exp ft,
LB -log Σ |EBu(s+} exp (cos f)/T)
(5)
Figure 3: Coarse-to- tinFine Learning of COFFTEA, In- batch learning (a) is first applied with coarse-grained in-batch negative examples In-candidate learning (b) is then employed to differentiate gold frame with hard negatives C.
exp (+)
Le -log Excou(s- exp exp (cos(4, f)/T)
(6)
3.2.1 In-batch Learning
Training directly w/ If may cause the model to primarily focus on distinguishing the differences between the gold frame and the candidates. In order to broaden the model's horizons, we first conduct training w/o If. allowing the model to learn to differentiate coarse coarse-grained semantic differences. We follow Khosla al. (2020) to construct in-batch negative examples B.
encoder, the frame encoder not only focuses on aligning frames with targets but is also intended to reflect the semantic relationships between frames themselves, such Inheritance (Su al., 2021; Zheng al., 2022).
3.2 Learning Process
order to capture the alignment between frames and targets, we aim to minimize the distance be- tween the target and the gold frame (positive pair) while maximizing the distance between the target and other frames (negative pairs) during the train- ing process. However, encoding such a large num- ber of frames, which exceeds 1000, and updating the encoders with gradients for all of them becomes infeasible. Therefore, we need to sample subset of frames as negative pairs. As discussed carlier, both training w/o If and w/ If have their limitations. To address this, we propose two-stage learning process in Fig. 3.
3.2.2 In-candidate Learning
Through in-batch learning, our model can effec- tively differentiate some frames that are relatively easy to distinguish. However, we aim to further distinguish frames that are more likely to be con- fused with one another. Typically, when using If, the other candidate frames for the same target can serve negative examples C.
However, the number of candidate frames for different targets can vary, and there are even cases where some targets only activate one frame FrameNet. To address this issue, we pad the num- ber of C for all targets to fixed number (e.g., 15). This padding ensures that each target has sistent number of candidate frames available for comparison during the training process.
We first employ in-batch learning ($3.2.1), where treat the gold frames of other targets in the same batch negative examples. This al- lows us learn coarse-grained frame semantic differences. Then we utilize in-candidate learning ($3.2.2), where we treat other frames in lexicon filter candidates as hard negatives. This step helps refine the frame representations and capture fine- grained semantic distinctions
We consider leveraging the semantic relation- ships between frames FrameNet ($A.3) to ex- pand C. Inheritance represents the "is-a" relation- ship between frames. For example, receiving in- herits from getting, then getting becomes the super frame and receiving becomes the sub frame. Other frames that also inherit from getting, such as Com- merce merce_buy and Taking, are considered siblings of receiving. These siblings exhibit semantic simi- larities with receiving, but with subtle differences, making them suitable as hard negative examples.
adopt cosine similarity cos(L, /) |لाााााा as distance metric and utilize contrastive learning distinguish positive pairs from negative pairs. Let Lg and Le denotes the contrastive loss of in-batch learning and in- candidate learning. The main difference between
Specificully, b.f

---

FN
Models
Ace
R@
R@3
R@5
Overall
Ace
R@l
R@3
R@5
Overall
Hartmann et al. (2017)f
87.63
77.49
82,25
83.00
76.10
79.40
KGFI (2021)f
92.13
85.63
88.76
92.40
85.81
91.59
92.88
88,98
FIDO (2021)f
91.30
92.10
Tamburini (2022)
92.57
92.33
Lookup Table
88.60
78.48
86.74
88.16
83,23
89.02
78.75
87.43
88.71
83.57
+ iuit. with Def
90.40
83.56
88.25
89.32
86.85
90,30
85.17
88.93
89.87
87.66
+ Ls
91.06
86.52
91.73
92.68
88.73
90.94
87.13
92.30
93.21
89.00
Ls onlyt
91.82
84.73
94.47
96.18
88.13
91.58
84.88
93.88
95.90
88,10
Le onlyt
92.50
76.51
87.35
91.01
83.75
92.70
78.33
88.93
91.64
84,91
La only
90.94
86.77
95.14
96.79
88.81
91.36
87.38
95.29
96.77
89.33
COFFTEA
92.55
87.69
92.64
94.71
92.64
87.34
92.91
94.29
89.91
Table 2: Main results on FN1.5 and 1.7. Acc denotes accuracy with lexcion filtering and R@k denotes recall at k without lexicion filtering. Overall represents the harmonic of Ace and R reflecting the comprchensive capability of models. Through dual encoders and two-stage learning, COFFTEA outperforms the previous methods (the topmost block) by at least 0.93 overall scores. Ablation study on dual encoders and two-stage learning listed in the middle and bottom blocks ($ 4.4). means the model does not trained with exemplar sentences.
We follow the order of candidate frames, sibling frames, and random frames to construct C.
4.2 Models
We mainly compare COFFTEA with representa- tive previous methods from the different cate- gories shown in Fig. 2: traditional representation learning based methods without definitions (Hart- mann et al., 2017), dual encoder architectures with frozen frame encoders (KGFI (2021), Tamburini (2022)), and single fused encoder architectures (FIDO (2021)).
4 Experiment
Our experiments mainly cover three aspects:
Main experimental results on frame identi- fication: We compare the performance of COFFTEA with previous approaches both with lexicon filtering (w/ If) and without lexicon filtering (w/o If)
In order to conduct further ablation study and deeper analysis, we also implement two basclines represented (a) and (b) in Fig 2: Lookup Ta- ble with random initialization and Lookup Table initialized with definition representations. The implementation details of previous methods, base- lines, and COFFTEA can be found in $A.1.
Further ablation study: to demonstrate the con- tributions of the learnable frame encoder and Coarse-to-Fing Fine learning to the performance of COFFTEA
In-depth representation analysis: We delve into a deeper analysis to investigate how the frame and target encoder of COFFTEA can bet- ter represent the alignment between frames and targets, as well as the relationships be- tween frames and frames, targets and targets.
4.3 Main Results
To compare our model with previous approaches, as well as our baselines. We report the accuracy (Ace) when lexicon filtering is applied, recall at k R @ k) when no lexicon filtering is used (indicating whether the gold frame ranked in the top & based on similarity), and an Overall score (harmonic average of Ace and R@
4.1 Datasets
We have introduced COFFTEA on two FrameNet datasets: FN 1.5 and FN 1.7. FN 1.7 is an extensive version of FN 1.5, with more annotated sentences and more defined frames. Following Swayamdipta al. (2017); Su al. (2021), we spilt the full text into train, dev and test set, and further use ex- emplars FrameNet as pretraining data like Peng et al. (2018); Zheng et al. (2022); Tamburini (2022). Table 1 shows the statistics of two datasets.
The upper block of Table 2 shows the perfor- mance of previous representative methods in frame identification. COFFTEA achieves performance gain of 0.93 overall scores (88.98 v.s. 89.91), pri- marily due 1.53 increase in R@ scores (85.81 v.s. 87.34) in the w/o If setting. This improvement can be attributed to the use a learnable frame

---

Models
Predictions w/o If from rank to (Top-1 is Becoming)
Le onlył
Transition a Undergo transformation
Change post, state Make acquaintaaee
Undergo change
Le onlyi
Activity_stee _strtt
Launch_ process
Meet dleetsspecification
Table 3: Predictions positions to 5 of "The children families who come Pleasant Run are given the opportunity to become happy". red means the frame is irrelevant to the gold frame Becoming.
encoder and two-stage learning, which enhance the effectiveness of COFFTEA in modeling definitions. Compared to KGFI, COFFTEA achieves compara- ble Acc w/ If (92.40 v.s. 92.64) and consistently performs better in R@k, which further demon- strates that through combining in-batch learning and in-candidate learning, COFFTEA is able to se- lect the uppropriate frame from both candidates (w/ If') and all frames (w/o If)
ences, resulting a significant increase of 1.70 points in accuracy w/ If (90.94 v.s. 92.64). This demonstrates the effectiveness of incorporating an unfrozen frame encoder to enhance the modeling of subtle semantic distinctions and improve the performance of the frame identification task.
Coarse-to-Fine Learning The bottom block Table 2 demonstrates the trade-off between in-batch learning (Ls only) and in-candidate learning (Lc only). LB only achieves the highest R @@) (84.88) and Le only yields the highest Acc (92.70). How- ever, the former approach does not excel at distin- guishing fine- fine-graine grained frames (91.58 Acc), while the latter approach even performs worse than the lookup table R@ w/o (78.33 v.s. 78.75).
Moreover, it is worth noting that some methods (marked t) does not utilize exemplars training data, while previous researches (Su al., 2021; Zheng et al., 2022) believe that using exemplars for training improves aceuracy. Further discussion about using exemplars can be found in $4.4.
In addition to representative methods in Table 2, there is series of frame identification studies that solely focus on Acc w/ If. We report the results of COFFTEA and more previous frame identification works in $B.2.
Table 3 gives a specific example for compari- son. Both methods can correctly predict the frame corresponding to the target become in the given sentence. However, when looking the predicted frames in positions 2 to 5, the frames predicted by in-batch learning show structural relationships with Becoming, while the frames predicted by in- candidate learning have no connection to becoming. This reflects that in-candidate learning cannot dis- tinguishing frames sufficiently.
4.4 Ablation Study
Frame Encoder Can learnable frame encoder contribute to frame identification? As is shown in the middle block in Table 2, lookup table without definitions achieved relatively low overall scores (83.57). However, when we initialize the lookup table with the definitions encoded by the frozen encoder, the lookup table showed significant im- provement in R@ w/o If, with an increase of 6.42 points (78.75 v.s. 17). This indicates that even without adjusting the parameters, the frozen en- coder of PLMs capable of modeling the defini- tions to some extent.
To address this issue, we train COFFTEA with Lg on exemplars and then train COFFTEA with Le with train split, which is equivalent to performing coarse- parsee-raiied pretraining on a large-scale corpus and then fine-tuning on a smaller dataset for do- main adaptation at finer granularity.
We also apply in-batch learning both train and exemplar. Due to the lack of hard negatives, its performance on Acc w/ If is lower than COFFTEA 191.36 v.s. 92.64).
To further explore its impact, we apply in-batch learning to the lookup table with definitions. This additional step results in a further improvement of 1.96 points in R 1 w/o (85.17 17 v.s. 87.13). This suggests that in-batch learning reduces the number of negative samples, decreases the learning diffi- culty, and ultimately leads to better performance. Comparing COFFTEA to these lookup tables, we find that unfreezing the frame encoder allows for better modeling of fine- ore-neai semantic differ-
Exemplars Using exemplars as additional train- ing data is common practice (Peng et al., 2018; Chen et al., 2021; Zheng et al., 2022). Since in-batch learning benefits from larger-scale data (84.88 v.s. 87.38 in Table 2), chose use the larger set, exemplars for the first stage of in in-batch learning. However, this makes our results not di- rectly comparable to those of previous methods (Su

---

Models
exem.
exem.
A
10 target exemplars
Models
R@0 1
Tamburini (2022)
91.42
92.33
-0.91
R@l
R@3
R@5
Lookup Table
89.02
89.41
-0.39
Lookup Table
78.75
63.96
75.54
77.27
init. with Def
90.40
90.66
-0.26
init. with Def
85.17
81.71
89.51
90.33
Ls
91.06
91.23
-0.17
LB
87.13
83.04
92.24
93.40
Ls only
91.36
91.58
-0.22
COFFTEA
87.34
85.58
93.07
94.89
COFFTEA
92.55
92.24
0.31
Table 6: Representing frames with definitions and ten target exemplars. The target encoder of COFFTEA can get similar and consistent representations of targets lated to the same frame
Table 4: Training with extra exemplans does not increase the model performance. COFFTEA can effectively uti- lize exemplars via two-stage learning.
Models
Average Aa/a
Models
Acc
Ace w/ [MASK]
A
Lookup Table + init. with Def
0.158
Lookup Table
89.02
78.11
10,92
0,012
init. with Def
90.30
77.58
12.72
+ LB
0,070
Ls
90.94
77.91
13.03
Ls onlyf
56,290
COFFTEA
92.64
85.82
6.82
Le onlyf
6.494
Table 5: Masking target words can cause the decrease of accuracy. The extent of the decrease, A, reflects whether the model understands the context information of the target.
COFFTEA
121.816
COFFTEA w/o. sibling
34.907
Table 7: The learnable frame encoder better models the structured information between frames. The perception of structure in COFFTEA comes from the first stage of in batch learning and the utilization of siblings as hard negatives.
al., 2021) that only use the training set. However, more training data does not necessarily lead to im- proved performance. As shown in Table 4, the per- formance Tamburini (2022) and our lookup table baselines decreases after using exemplars, possibly due to domain mismatch between the exemplars and the training set. However, our two-stage ap- proach effectively utilizes both datasets to get improvement of 0.31 Acc, providing more appro- priate way to utilize exemplars for future work.
encoder structure to effectively model the align- ment between target context and frames. This ca- pability also indicates that COFFTEA explore out-of-vocabulary targets.
The target encoder en- sure a close proximity between targets and their corresponding frames. However, we believe that different sentences the target representations in dif- ferent sentences, which activate the same frame, are supposed to exhibit high degree of similar- ity. We seleet ten exemplars for each frame and replace frame representations with the mean repre- sentation (centroid) of these exemplars. In Table 6, we compare two frame representations: directly en- coding the definition and encoding ten exemplars. COFFTEA achieves consistent and similar perfor- mance in both settings (87.34 and 85.58), reflecting the similarity between the target representations of the same frame and the final frame representation. as captured by our target encoder.
Target-Targer Relations
4.5 Further Analysis
Target-Frame Alignments The results in Ta- ble demonstrate that COFFTEA effectively models the alignment between targets and frames. How- ever, still want explore whether the model's ability predict a frame is due its accurate mod- eling of the target's contextual information in the sentence or if there are other shortcuts involved. Moreover, if COFFTEA can effectively model the context of the target, can also be used to dis- cover voccblluyya targets. To investigate this, we replace the target in the sentence with the [MASK] token and compute the similarity between the masked target representation and frames. Ta- ble 5 shows that when the target masked, the accuracy of the model decreases across the board. COFFTEA achieves the smallest decrease in perfor- mance (A), indicating the capability of our dual-
Frame-Frame Relations Due the semantic re- lations defined by FrameNet between frames (such as inheritance, using, subframe, etc.), frames are not isolated but rather interconnected with each other, forming structural relationships. For exam-

---

ple, if receiving inherits from getting, their repre- sentations should be similar. Frame representations should reflect the structural relationships. To eval- uate the degree of similarity between a subframe Suu and its superframe Ssup- we define a metric,
propose fused encoder evaluate semantic coher- ence between sentence and frame definition. Nev- ertheless, it just fails to evaluate the performance of w/o If scenario.
In addition, the aforementioned studies tend to adopt one-sided relationship between targets and frames. They only train model on either w/ If or w/o lf, which can not catch an overall horizon in distinguishing frames. Moreover, only Su et al. (2021) release performance under w/o If scenario, which is essential for evaluating the full capabilities of the model.
Σ
cos(Ssub: f) FEF
Aa = cos (Jsub: Ssuu) a
(8)
Ae is normalized evaluation reflecting how subframe is close its superframe compared to other frames. We report the average da (a 0) of inheritance pairs in Table
Some other work like Chanin (2023) treat frame semantic parsing as a sequence-to-sequenc text generation task, but its still trap in training under w/ If and can not catch an optimal relationships.
The baselines (the upper block) without a learn- able frame encoder struggle to model the relation- ships between frames effectively. The distance between subframes and superframes is almost the same that between other frames. However, us- ing learnable frame encoder greatly improves the similarity between subframes and superframes.
Metric Learning Metric learning is a fundamen- tal approach in machine learning that seeks to learn a distance metric or similarity measure that accurately captures the inherent relationships be- tween data points. One particular technique that has gained prominence in this area is Contrastive Learning, which has been extensively applied variety of tasks such text classification (Gao et al., 2021), image classification (He et al., 2020).
In-candidate learning, if used alone, tends to overly focus on differentiating frames with simi- lar semantics, which actually decreases the simi- larity between them (6.494 v.s. 56.290). On the other hand, COFFTEA effectively models the sim- ilarity between associated frames. This is mainly because we use two-stage learning and treat sib- lings hard negative examples during training (34.907 v.s. 121.816), which indirectly enables the model learn the connection between subframes and superframes.
The optimized feature space retrieved from Con- trastive Learning objective can benefit on learning better representations of data and capturing inner relationship. For frame identification, the primary goal is to obtain a well-aligned target-to-free frame space. However, existing researches only focus using Cross Entropy to directly modeling cor- relation to All Frames, which presents challenges in capturing complex relationships. Alternatively, Contrastive Learning can be leveraged to optimize and incrementally construct the feature space of frames and targets.
5 Related Work
Frame Identification Prior research in frame identification has primarily focused on modeling the similarity between targets and frames or assess- ing the matching score between them.
In terms of modeling similarity, Hartmann et al. (2017): Peng al. (2018); Zheng al. (2022) lever- age target encoder with trainable frame represen- tation table, while ignoring the valuable contextual frame information provided by FrameNet. To rem- edy this deficiency, Su et al. (2021); Tamburini (2022) both propose to incorporate frame defini- tions into frame encoder. Despite these efforts, the number of frames and the optimizing objective make it infeasible update frame representation during training, leading to compromise of freez- ing Consequently, they all trap the dilemma to obtain optimal frame representation. As for as- sessing the matching score, Jiang and Riloff (2021)
Conclusion
We propose COFFTEA, and its dual encoder can effectively model the definitions with learnable frame encoder and efficiently calculate the simi- larity scores between a target and all frames. We also conclude that there exists a trade-off between training w/ If and w/o If. and achieves comprehen- sive performance with a two stage learning process. Detailed analysis of the experiments further demon- strates that our two encoders are not limited to their alignment but can also discover other connections between frame and frame, as well as target and target.

---

Limitations
Jacob Devlin, Ming-Wo Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre- tre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association For Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pages 1171,4186, Minneapolis, Minnesota. Asso- ciation for Computational Linguistics.
The limitations of our work include:
1. We only used cosine similarity distance metric and did explore other methods of metric learning. Investigating alternative dis- tance metries could potentially enhance the performance of our model.
Charles Fillmore et al. 1976. Frame semantics and the nature language. In Annals of the New York Academy of Sciences: Conference on the origin and development of language and speech, volume 280. pages 20-32. New York.
2. Currently, the FrameNet dataset does not provide test data for out-of-vocabulary words. Therefore, our exploration of out-of- vocabulary words is limited to preliminary experiments (Table 5). Further investigation with dedicated dataset containing out-of- vocabulary test data would be valuable.
Tianyu Gao, Xingeheng Yao, and Danqi Chen. 2021. SimCSE: Simple contrastive learning of sentence em- beddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894-6910, Online and Punta Cana, Domini- can Republic. Association for Computational Lin- guistics.
3.
We only applied the dual-encoder architecture to frame identification and did not explore its application to frame semantic role labeling. Extending our approach to frame semantic role labeling would provide insights into its ef- fectiveness and generalizability beyond frame identification.
Daniel Gildea and Daniel Jurafsky. 2000. Auto- matic labeling of semantic roles. In Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics. pages 512-520, Hong Kong. Association for Computational Linguistics.
Shaoru Guo, Yong Guan, Ru Li, Xiaoli Li, and Hongye Tan. 2020. Incorporating syntax and frame seman- tics in neural network for machine reading compre- hension. In Proceedings of the 28th International Conference on Computational Linguistics, pages 2635-2641, Barcelona, Spain (Online). International Committee on Computational Linguistics.
Acknowledgements
We thank all reviewers for their great efforts. This paper is supported by the National Science Foun- dation of China under Grant No.61936012.
Silvana Hartmann, Kuznetsov, Teresa Martin, and Iryna Gurevych. 2017. Out-of-domain FrameNet semantic role labeling. In Proceedings of the 15th Conference the European Chapter the Association for Computational Linguistics: Volume 1, LLong Papers, pages 47 171-482, Valencia, Spain. As- sociation for Computational Linguistics.
References
Collin Baker, Michael Ellsworth, and Katrin Erk. 2007. SemEval-2007 task 19: Frame semantic structure extraction. Proceedings of Fourth International Workshop on Semantic Evaluations 2000), pages 19-104, Prague, Czech Re- public. Association for Computational Linguistics.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729- 9738.
Collin F. Baker, Charles J. Fillmore, and John B. Lowe. 1998. The Berkeley FrameNet project. 36th Annual Meeting the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume pages 86-90, Montreal, Quebec, Canada, Associa- ton for Computational Linguistics.
Tianyu Jiang and Ellen Riloff. 2021. Exploiting def- initions for frame identification. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2429 2434, Online. Association for Computational Linguistics.
David Ian Chanin. 2023. Open-source frame semantic parsing. ArXiv, abss22033.1288..
Xudong Chen, Ce Zheng, and Baobao Chang. 2021. Joint multi-decoder framework with hierarchical pointer network for frame semantic parsing. In Findings of the Association for Computational Linguisties: ACL-IICNLP 2021, pages 2570-2578, Online. Association for Computational Linguistics.
Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishman. 2020. Su- pervised contrastive learning. Advances in neural information processing systems, 1

---

Hao Peng, Sam Thomson, Swabha Swayamdipta, and Noah A. Smith. 2018. Learning joint semantic parsers from disjoint data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long Papers), pages 1492-1502, New Orleans, Louisiana. Association for Computational Linguistics.
Nils Reimers and Iryna Gurevych. 2019. Sentence-
BERT: Sentence embeddings using Siamese BERT- networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural I. Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-ICCNLP) pages 3982-3992. Hong Kong. China. Association for Computational Linguistics.
Yuqi Si and Kirk Roberts. 2018.
frame-based nip system for cancer-related information extraction. In AMIA annual symposium proceedings. volume 2018, page 1524, American Medical Informatics As- sociation.
Xuefeng Su, Li, Xiaoli Li, Jeff Z. Pan, Zhang, Qinghua Chai, and Xiaoqi Han. 2021. A knowledge- guided framework for frame identification. In Proceedings of the 59th Annual Meeting of Association for Computational Linguistics and the Natural Language Processing (Volume 1: Long Papers), pages 5230-5240, Online. Association for Computa- tional Linguisties.
11th International Joint Conference
Swabha Swayamdipta, Sam Thomson, Chris Dyer, and Noah A Smith. 2017. Frame-semantic parsing with daftmax-marghh segmental mns and a syntactic scaf- fold. arXiv preprint #riv::7706005528
Fabio Tamburini. 2022.
Combining ELECTRA and adaptive graph encoding for frame identification. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 1671-1679, Mar- seille, France. European Language Resources Asso- ciation.
Ce Zheng, Xudong Chen, Rumxin Xu, and Baobao Chang. 2022. A double-graph based framework for frame semantic parsing. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4998-501 Seattle, United States. Association for Computational Lin- guistics.
Ce Zheng, Yiming Wang, and Baobao Chang. 2023. Query your model with definitions framenet: an effective method for frame semantic role labeling. In Proceedings of the AAAI Conference Artificial Intelligence, volume 37, pages 14029-14037.

---

A Experiment Details
Hyper-parameter
Value
Epochs
20
A.1 Baseline Models
Learning Rate
2e-5
Optimizer
Adam
Batch Size of La
32
Hartmann et al. (2017): a simple classifier based on the representation of entire sentence and espe- cially dependents of target.
Gradient Accumulation of Ls
4
Temperature of Ls Batch Size of Le
0.07
Gradient Accumulation Le Candidate Frame Number Le Temperature Le Dual Encoders
Peng et al. (2018): joint model learn semantic parsers from disjoint corpora with latent variable formulation.
BERT-base
Chen et al. (2021): a joint model of multi-decoder and hierarchical pointer network for frame seman- tic parsing.
Table 8: Hyper-parametee settings
Sup frame
Sub frame
FIDO (2021): a fused model assessing semantics coherence between sentence and frame definition. KGFI (2021): dual encoders model for frame identification aligning similarity of targets and pre- computed frame representation.
Receiving
Amassing
Getting
Commerce
Commerce_colee __cllectttttttteeeeeeeeee
Taking
Receiving
Borrowing
Tamburini (2022): similar to Su et al. (2021) with more reliable graph technique.
Becoming
Transition to state
Undergo change
KID (2022): a double graph based model for frame semantic paring.
Undergo_change
Bandergo_transtornnn
AGED (2023): a qquery-based framework for Frame Semantic Role Labeling under frame definition. Chanin (2023): T5-based model treating frame semantic parsing as sequence-to--eqqence text gen- eration task.
Becoming
Transition to quality
part of pairs of Inheritance rela- tion, see more at https://framentt icsi berkeley. edu/Fndrupal/.
Table
Lookup Table: a baseline model aligning target encoder with trainable one-hot mapping frame rep- resentation.
B Supplementary Experiment Results
B.1 Statistical Results of COFFTEA
Lookup Table init. with definition: baseline model aligning target encoder with trainable BERT- initialized frame representation.
To verify that experimental results not accidental, we train COFTEEA with five different random seeds for each dataset. Compare with the strongest model (Su et al., 2021), we can get bet- ter performance consistently. Table 10 shows the average performances with its deviation.
Lookup Table init. with definition and Lg: using Contrastive Loss as optimize objective instead of Cross Entropy.
A.2 Hyper-parameter Setting
Avg. + Dev.
Dataset
Models
w/ If
w/o
For replicability of our work, we list the Hyper- parameter setting of all our baselines and COFFTEA in Table 8.
KGFI
92.40
84.41
FN1.5
COFFTEA
92.51 1: 0.14
87.26 1: 0.10
KGFI
92.13
85.63
FNI.7
COFFTEA
92.50 + 0.05
87.29 + 0.41
A.3 Frame Relation
Table 10: Statistical results of multiple runs on FN1.7. We train models in with five different random seeds and report the average performance with deviation.
Given frame f. get its sibling frames Faa. utilize the frame relation Inherilance and con- sider f the superior Inheritance to retrieve its subordinate or superior frames, and then get the sibling of f. Table 9 shows the Inheritance relation we used.
subordinate frame of
B.2 Comparison with More Methods
As most previous work only focus on performance in the w/ If setting, we conduct a comprehensive

---

comparison both All frames and Amb frames, where Amb subset of All whose target is poly- semous or can evkoe multiple frames. And we find that Tamburini (2022) just use a smaller test set on both FN1.5 and FN1.7. So our COFFTEA achieves competitive performance compared to the strongest models.
FN 1.5
FN 1.7
Models
All
Amb
All
Amb
Hartmann et al.
87.63
73.80
83.00
71.70
Peng et al.
90.00
78.00
89.10
77.50
Chen et al.
90.50
79.10
FIDOF
91.30
81.00
92. 10
83,80
KGFI
92.13
82.34
92.40
84.41
AGED
91.63
81.57
KID
91.70
81.72
91.70
83.03
Tamburini
Chanin 92.57
83.58
92.33
84.22
89.00
77.50
COFFTEA
92.55
84.95
Table 11: Frame identification accuracy with lexicon fil- tering on FrameNet test dataset. 'All' and 'Amb'' denote testing test data and ambiguous data respectively. means the model does not trained with exemplar sen- tences.


Reviews:
[{"review_id": "T6PD9cFK47", "reviewer": "Reviewer_wf6g", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We really appreciate your effort in reviewing our paper, and thank you for your helpful comments. We are glad to see that you recognize the rationality of our motivation and the effectiveness of our method. Now we address your questions below:  \n\nQ1: The proposed method lacks clear validation of the mutual benefits resulting from the two-stage training approach, particularly in terms of specific metrics such as ACC. \n\nA1: We really appreciate your interest in the mutual benefits of our two-stage learning and would like to address your concerns. In Table 1, we list the results of our each stage, where **$L_B$ performs better on w/o. LF (84.88 v.s. 78.33) while  $L_C$ performs better on w. LF (92.70 v.s. 91.58)**. To clarify this phenomenon, we give an example of the prediction result in Table 3, the frames predicted by **$L_B$ show structural relationships with golden frame, while $L_C$ just has no connection**. This indicates that **no matter which stage is used alone, the alignment learned is insufficient**, but our COFFTEA realizes to combine the advantages and achieves the best overall performance with 0.93 improvement.\n\nIn addition to using quantitative indicators like ACC, we also designed a set of experiments to demonstrate the mutual benefits of COFFTEA in Section 4.5. One of them demonstrates that **our model can better learn the relation between frames**. We define a metric refer to Eq 7 to measure the degree of similarly between a subframe and its superframe compared with other frames. In Table 7, we show that the baselines without a trainable frame encoder struggle to distinguish all frames, with a degree about 0.100, while **COFFTEA can surprisingly simulate the role of two stages and get a score of 121.816**, indicating COFFTEA effectively models the similarity between associated frames.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper provides a clear motivation by highlighting the advantages and disadvantages of the widely-used lexicon filtering (LF) strategy. Consequently, a corresponding method is proposed to further enhance learning with LF.\n\n2. Each component within the proposed framework is both effective and straightforward, offering promising prospects for replication with the provided code.\n\n3. The paper is well-written and predominantly easy to comprehend.", "reasons_to_reject": "There are only a few minor issues that need to be addressed:\nThe proposed method lacks clear validation of the mutual benefits resulting from the two-stage training approach, particularly in terms of specific metrics such as ACC. For instance, on the FN1.7 dataset, the overall framework (COFFTEA) demonstrates slightly lower performance compared to $L_{C}$ only†. It would be helpful to provide further clarification on this matter.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper seeks to enhance the correspondence between linguistic words and video frames in the context of frame identification. Recognizing the ambivalent nature of the commonly employed lexicon filtering (LF) strategy, this study introduces a two-stage learning framework that initially incorporates LF and subsequently omits it. Furthermore, a dual encoder structure is adopted within this framework. Through comprehensive experimentation, the effectiveness of the proposed components within the framework is validated.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "DXkarh0kt6", "reviewer": "Reviewer_9bh3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 1, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We really appreciate your effort in reviewing our paper, and we are glad to see that you recognize our writing and motivation. We will proofread and revise our typos carefully.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "I am not the expert in the topic, but I think the paper is very easy to read.\nThe motivation is clear, and the proposed solution shows reasonable for addressing the problem.\n", "reasons_to_reject": "Some typos should be revised.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes a COFFTEA, a Coarse-to-Fine Frame and Target Encoders Architecture. By contrastive learning  and dual encoders, COFFTEA efficiently and effectively models the alignment between frames and targets. With a coarse-to-fine curriculum learning procedure, the proposed method gradually learns to differentiate frames with varying degrees of similarity.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "758sPmiAvV", "reviewer": "Reviewer_nLkN", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We really appreciate your effort in reviewing our paper, and thank you for your helpful comments. We are glad to see that you recognize 1) our model efficiently align target with all frames 2) our method well-motivated and effective. Now we address your questions below: \n\nQ1: The overall dual-encoder framework and contrastive learning are lack of novelty. There is no new methodology proposed and the framework described is basically incremental.\n\nA1: In this paper, **our innovation depends on our focus on and analysis of the two overlooked issues in Frame Identification  - lexicon filtering and insufficient learning**. Ideally, we can use contrastive learning to perfectly align target with all frames and dual-encoder framework to sufficiently promote effective learning of frames.\n\nAt the same time, we precisely design methods to prove that our two-stage learning is mutually reinforcing, rather than simply piling up. Our three parts in Section 4.5 respectively demonstrate that:\n1) we learn **good alignments between targets and frames**, where we get SOTA performance on two benchmarks and both scenarios.\n2) we **obtain better target representation which fully understand the contextual context**. In Table 6, when using the learned exemplars' target representation to recall frames, **COFFTEA get 85.58, the closest to w/o. lf 87.34**.\n3)  our **frame representations rich in frame relationships**, which enable us to maximize the differentiation between subframes with inheritance relationships relative to superframes and other frames, **with highest scores of 121.816**.\n\nQ2: The paper claims the ability of addressing out-of-vocabulary targets, but no experimental analysis is provided.\n\nA2: As our limitation states, currently, the FrameNet dataset does not provide test data for out-of-vocabulary words. Therefore, our exploration of out-of vocabulary words is limited to preliminary experiments. **Based on the assumption of word distribution, words with the same semantics often appear in the same context**. Therefore, **by masking the corresponding target, we can simulate our model's ability to match the corresponding frame when out-of-vocabulary targets appear**. In Table 5, when target is masking, **our COFFTEA declines least by 6.82% compared to the baselines decrease above 10%**, indicating our strongest potential to handle out-of-vocabulary targets.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper is well written and easy to follow.\n- Dual encoder structure allows efficiently computing similarity between target and all frames.\n- The proposed coarse-to-fine learning addresses limitations of only training with or without lexicon filtering.\n- The method achieves strong performance on frame identification benchmarks, especially without lexicon filtering.", "reasons_to_reject": "- The overall dual-encoder framework and contrastive learning are lack of novelty. There is no new methodology proposed and the framework described is basically incremental.\n- The paper claims the ability of addressing out-of-vocabulary targets, but no experimental analysis is provided.", "questions_to_authors": "", "paper_topic_and_main_contributions": "- Proposes a dual encoder model  for frame identification using contrastive learning.\n- It adopts dual encoder structure for efficiency.\n- It introduces a two-stage coarse-to-fine learning procedure to handle trade-off between training with and without lexicon filtering.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: In this work, the authors tackle the longstanding CL task of frame identification (matching of words in context with semantic frames they induce) and propose an approach based on reshaping of a representation space to relfect the topology of frame-based similarity.

The approach, in principle very similar to approaches that convert language models into sentence encoders, consists of: (1) dual encoding (separate encoding of the context of the target word and the sentential example(s) that instantiate the frame); (2) contrastive learning (contrastive loss effectively reshapes the encoder so that the proximity of vectors of sentential instances reflects their alignment in terms of the semantic frame they correspond to, i.e., which they induce); and (3) coarse-to-fine curriculum learning in which the difficulty of negative examples is gradually increased (i.e., one starts with randomly selected, and likely very different semantic frames as negatives in early stages of training and then progresses to use semantically closer frames as negatives in later stages). The authors show empirically that all three components are important for the reported performance gains.   

All reviewers agree that the work is sound and identify no errors in evaluation protocols. The proposed approach is meaningful and yields gains, although the gains are not particularly large. The reviewers -- and I fully support this view are somewhat more skeptical of the methodological novelty: all three components, namely, the bi-encoding, contrastive learning, and curriculum with increasingly difficult negatives are very widely and commonly used techniques, widely used in sentence-level classification and ranking tasks. From this point of view, the work can be seen as an application of this well-known framework to the task of frame identification (where instances are also sentences or quasi-sentences). 

Overall, I find this work to be sound and valuable for the CL community and the narrower community that focuses on frame identification, but not particularly exciting for the wider NLP audience.
Decision: Accept-Findings