Generating Lexical Representations of Frames using Lexical Substitution (2020)

Abstract: Semantic frames are formal linguistic structures describing situations/actions/events, e.g. Commercial transfer of goods. Each frame provides a set of roles corresponding to the situation participants, e.g. Buyer and Goods, and lexical units (LUs) – words and phrases that can evoke this particular frame in texts, e.g. Sell. The scarcity of annotated resources hinders wider adoption of frame semantics across languages and domains. We investigate a simple yet effective method, lexical substitution with word representation models, to automatically expand a small set of frame-annotated sentences with new words for their respective roles and LUs. We evaluate the expansion quality using FrameNet. Contextualized models demonstrate overall superior performance compared to the non-contextualized ones on roles. However, the latter show comparable performance on the task of LU expansion.

## introduction the goal of lexical substitution @xcite is to replace a given target word in its context with meaning-preserving alternatives. in this paper, we show how lexical substitution can be used for semantic frame expansion. a semantic frame is a linguistic structure used to describe the formal meaning of a situation or event @xcite . semantic frames have witnessed a wide range of applications; such as question answering @xcite @xcite , machine translation @xcite , and semantic role labelling @xcite . the impact, however, is limited by the scarce availability of substitutes for assistance: assist, aid substitutes for helper: she, i, he, you, we, someone, they, it, lori, hannah, paul, sarah, melanie, pam, riley substitutes for benefited party: me, him, folk, her, everyone, people substitutes for time: tomorrow, now, shortly, sooner, tonight, today, later we suggest a different perspective on the problem: expanding the framenet resource automatically by using lexical substitution. given a small set of seed sentences with their frame annotations, we can expand it by substituting the targets (words corresponding to lexical units of the respective frame) and arguments (words corresponding to roles of the respective frame) of those sentences and aggregating possible substitutions into an induced frame-semantic resource. to generate substitutes, we decompose the problem into two sub-tasks: lexical unit expansion: given a sentence and its target word, the task is to generate frame preserving substitutes for this word. frame role expansion: given a sentence and an argument, the task is to generate meaningpreserving substitutes for this argument. contributions of our work are (i) a method for inducing frame-semantic resources based on a few frame-annotated sentences using lexical substitution, and (ii) an evaluation of various distributional semantic models and lexical substitution methods on the ground truth from framenet. ## related work approaches to semantic frame parsing with respect to a pre-defined semantic frame resource, such as framenet, have received much attention in the literature @xcite @xcite , with semafor @xcite being a most widely known system to extract complete frame structure including target identification. some works focus on identifying partial structures such as frame identification @xcite , role labelling with frame identification @xcite , and simple role labelling @xcite @xcite , which is considered very similar to standard prop-bank @xcite style semantic role labelling, albeit more challenging because of the high granularity of frame roles. these supervised models rely on a dataset of frame-annotated sentences such as framenet. framenet-like resources are available only for very few languages and cover only a few domains. in this paper, we venture into the inverse problem, the case where the number of annotations is insufficient, similar to the idea of @xcite who investigated the utility of semantic spaces and wordnet-based methods to automatically induce new lus and reported their results on framenet. our method is inspired by the recent work of @xcite . they suggest to predict the substitutes vectors for target words using pre-trained elmo @xcite and dynamic symmetric patterns, then induced the word senses using clustering. @xcite takes the idea of substitute vectors from @xcite for the semeval 2019 @xcite frame induction task and replaces elmo with bert @xcite for improved performance. zhou et al. (2019) show the utility of bert for the lexical substitution task. lexical substitution has been used for a range of nlp tasks such as paraphrasing or text simplification, but here, we are employing it, as far as we are aware, for the first time to perform expansion of frame-semantic resources. ## inducing lexical representations of frames via lexical substitution we experimented with two groups of lexical substitution methods. the first one use no context: non-contextualized neural word embedding models, i.e. fasttext @xcite , glove @xcite , and word2vec @xcite , as well as distributional thesaurus based models in the form of jobimtext (biemann and riedl, 2013). the second group of methods does use the context: here, we tried contextualized word embedding model bert @xcite and the lexical substitution model of @xcite . ## conclusion we presented a simple practical technique for the generation of lexical representations of semantic frames using lexical substitution with several contextualized and static word representation models demonstrating that a single frame annotated example can be used to bootstrap a fully-fledged lexical representation of the framenet-style linguistic structures. non-contextualized baseline models proved to be strong baselines, but failed to produce good substitutes for polysemic words (same word but different semantic frame), whereas bert for such cases produced competitive substitutes. a prominent direction for future work is testing the proposed technology for building frame representations of low-resource languages and domains.