Injecting Frame Semantics into Large Language Models via Prompt-Based Fine-Tuning (2025)

Abstract: Large Language Models (LLMs) have demonstrated remarkable generalization across diverse NLP tasks, yet they often produce outputs lacking semantic coherence due to insufficient grounding in structured linguistic knowledge. This paper proposes a novel method for injecting Frame Semantics into a pretrained LLaMA model using Low-Rank Adaptation (LoRA). Leveraging FrameNet (a rich resource of over 1,000 semantic frames) we construct a training corpus comprising structured triples of frame definitions, frame elements, and lexical units. Our method encodes these examples into the model via LoRA adapters and evaluates performance using zero-shot prompting for textual entailment and semantic role labeling (SRL) over Framenet. Experimental results show that our adapted frame-aware LLM substantially outperforms the baseline across closed, open-ended, and multiple-choice prompts. Moreover, we observe significant improvements in SRL accuracy, demonstrating the efficacy of combining frame-semantic theory with parameter-efficient pretraining.

## introduction large language models (llms) such as gpt-4 @xcite and llama @xcite have demonstrated impressive capabilities across a wide range of natural language processing (nlp) tasks. however, despite their generalization strength, these models often lack explicit grounding in linguistic theories, which can occasionally result in fluent outputs that overlook deeper semantic distinctions and, in some cases, lead to factual inconsistencies or semantic hallucinations @xcite . to address this gap, enriching llms with structured linguistic knowledge could certainly be beneficial, as improved interpretability may support more reliable and semantically coherent outputs. one promising direction is the integration of frame semantics @xcite , a linguistic theory that connects word semantics to situational, i.e. conceptualized, information in terms of frames. each frame consists of a situation (i.e. the frame) and prototypical participants, known as frame elements (fes). it is triggered by specific lexical units (lus) in the text. for example, the verb provide triggers a supply frame, with roles such as supplier, recipient, and theme. unlike purely distributional approaches, frame semantics imposes situational constraints on semantic role assignments, grounding language interpretation in real-world scenarios. framenet @xcite , a computational resource based on frame semantics, offers a comprehensive repository of over 1,000 frames and their annotated instances. in fact, equipping models with frame-level information can make a tangible difference for tasks like semantic role labeling @xcite , question answering @xcite , and even commonsense reasoning @xcite . in this work, we present a parameter-efficient method to inject frame semantics into large language models (llms) through fine-tuning with low-rank adaptation (lora) @xcite . our central idea is to make an abstract linguistic theory usable by llms by textualizing its core concepts: we systematically convert framenet's structured knowledge (frame definitions, frame elements or fes, and lexical units or lus) into natural language examples in the form of question-answer pairs. for instance, we generate prompts that ask for the definition of a frame, the roles it involves, or the words that evoke it, thus producing an artificial dataset that "translates" theoretical content into a format suitable for instruction-based adaptation. in total, this process yields a dataset of 6,628 question-answer pairs covering 60 framenet frames. by fine-tuning llama models on this textualized dataset, we aim to encourage the model to internalize frame-semantic structures and relationships, enabling it to better reason about frames, roles, and their instantiations in text, even in the absence of explicit annotation. a critical question, however, is whether llms fine-tuned on such examples merely memorize specific facts about the frames encountered during training, or whether they actually generalize framesemantic knowledge to novel, previously unseen frames. to address this, we explicitly evaluate model performance on both seen frames (included in fine-tuning) and unseen frames (held out from training). this experimental design allows us to disentangle the model's ability to recall injected knowledge from its capacity to abstract and apply frame-semantic principles to new scenarios-an essential property for robust knowledge integration. we assess the effectiveness of our approach in two ways. first, we probe the model's framesemantic competence by evaluating its ability to answer structured questions about frames, elements, and lexical units-essentially measuring whether the injected knowledge is accessible via prompting. second, and more crucially, we test whether this knowledge generalizes to downstream tasks for which the model has not seen explicit training examples. in particular, we consider semantic role labeling (srl): given a sentence, can the model correctly identify and assign core frame elements? notably, during fine-tuning, the model is never shown labeled sentences (only definitions and conceptual relations) so improvements on srl reflect genuine semantic knowledge transfer. our results show that the frame-aware llm not only answers frame-related questions more accurately, but also outperforms the baseline on zero-shot srl tasks, supporting the claim that structured linguistic knowledge can be effectively injected via promptbased fine-tuning. our main contributions are as follows: 1) we propose a lightweight, lora-based method for injecting frame-semantic knowledge into llms using structured framenet annotations. 2) we design a diverse set of instructional prompting templates and linguistic variations to simulate realistic use cases for frame-role understanding. 3) we provide extensive evaluation on both seen and unseen frames for zero-shot knowledge probing and srl inference, demonstrating enhanced interpretability and generalization. in the rest of the paper, section 2 reviews re-lated work, section 3 describes our methodology, section 4 presents experiments and results, and section 5 concludes with final remarks and future directions. frame semantics, introduced by @xcite ), provides a principled approach to modeling linguistic meaning by organizing words into conceptual structures called frames. each frame represents a prototypical scenario, described by a set of frame elements (fes), and is evoked by specific lexical units (lus). the framenet project @xcite operationalizes this theory by cataloguing over 1,000 frames, their core and peripheral elements, and annotated instances. while large language models (llms) such as gpt-4 @xcite and llama @xcite achieve remarkable performance across diverse nlp tasks, they are pre-trained on general web corpora and lack explicit integration of structured linguistic resources like framenet. as a result, llms may generate fluent yet semantically misaligned outputs when required to interpret or generate language in terms of frame-semantic roles. recent research has sought to bridge this gap by augmenting llms with frame-semantic knowledge. fine-tuning llms on framenet data has been shown to enhance their ability to model semantic structures and improve interpretability @xcite . several works have proposed injecting frame-level information into transformer architectures to support semantic role labeling @xcite , question answering (madabushi, 2024), commonsense reasoning @xcite , and even named entity recognition @xcite . frame-based representations have also been leveraged for more robust and factually grounded summarization @xcite . despite these advances, most prior work either leverages framenet solely as a source of annotations for supervised tasks or incorporates frame information as static features. in contrast, our approach aims to internalize frame-semantic knowledge by textualizing framenet diverse knowledge into instructional prompts for llm adaptation. furthermore, we explicitly assess the quality of injected knowledge by evaluating the model not just on frames used for adaptation, but also on unseen frames held out from training, a perspective rarely addressed in prior studies. in summary, while previous research has demonstrated the benefits of integrating frame-semantic supervision into neural models, there remains a need for approaches that support robust generalization and interpretability via explicit, structured knowledge injection. our work aims to reduce this gap by proposing a scalable, prompt-based method for frame-semantic adaptation, and by providing a systematic evaluation on both in-domain and outof-domain (seen/unseen) frames. our work also relates to recent efforts in discourse semantics that employ question answering as a tool for evaluating consistency and logical understanding. for example, @xcite introduce a socratic qa framework to test whether llms respond consistently to logically equivalent or entailed discourse questions, while @xcite propose qudeval to measure semantic consistency across related qa pairs grounded in discourse theory. these studies highlight the importance of consistency in qa-based evaluation, which is complementary to our focus on injecting frame-semantic knowledge into llms. ## injecting frame-semantics into llms our knowledge injection pipeline, illustrated in figure 1 , is designed to make the structured content of framenet directly usable by large language models. the process begins with the extraction of frame-level information from framenet: for each frame, we collect its definition (a concise description of the scenario the frame represents), its core frame elements (the prototypical participants or roles involved), and the set of lexical units (words or multiword expressions that evoke that frame in context). to give a concrete example, rather than using framenet only as a source for supervised labeling, we transform this structured knowledge into a set of natural language question-answer pairs. for each frame, we generate prompts that ask about its definition, the roles it contains, or which words evoke it, simulating realistic queries a user or downstream application might pose. prompts are generated in various formats, including open-ended, closed-ended, and multiplechoice ensuring broad coverage of the theory. this synthetic qa dataset serves as supervision for fine-tuning a pretrained llama-3 model via low-rank adaptation (lora) @xcite . lora is a parameter-efficient fine-tuning technique that augments a frozen pretrained model with small trainable low-rank matrices. during training, only these additional parameters are updated, greatly reducing memory and computational cost while preserving the general linguistic competence already encoded in the model. these properties make lora particularly suitable for injecting structured resources like framenet, where the large number of frame-role combinations would make full fine-tuning both expensive and prone to catastrophic forgetting, i.e., the overwriting of previously acquired knowledge. by constraining learning to a compact set of additional parameters, lora enables the integration of frame-semantic knowledge without erasing the model's broader abilities. in our experiments, we adopt both 3b and 8b llama models, which balance computational feasibility with meaningful evaluation. this allows the model to internalize the relationships between frame definitions, roles, and lexical units, without relying on explicit sentence-level annotation. after training, the resulting frame-aware llm can be probed on zero-shot tasks such as semantic role labeling and frame-related question answering. ## experimental evaluation in this section, we evaluate the effectiveness of our frame-semantic knowledge injection approach for large language models (llms). our experimental objectives are twofold: (1) determine whether the injected knowledge substantially enhances the model's ability to reason about frames, frame elements, and lexical units; and (2) assess whether this acquired semantic knowledge generalizes effectively to practical downstream tasks, most notably, semantic role labeling (srl)-even without explicit srl supervision during training. ## conclusion and observations in this work, we introduced an efficient and principled methodology for injecting structured framesemantic knowledge into large language models via lora-based fine-tuning. by transforming framenet resources into instructional prompts, we enabled the model to internalize rich semantic abstractions grounded in linguistic theory. our experiments demonstrate substantial gains in both frame and role recognition tasks, as well as in zeroshot semantic role labeling (srl). importantly, the model exhibits strong generalization to unseen frames, highlighting its ability to abstract beyond surface-level associations and apply learned structures in novel contexts. these findings suggest that explicitly aligning llms with frame semantics can meaningfully enhance their semantic behavior, without sacrificing general language capabilities. this opens promising avenues for future research, including scaling to broader frame inventories, by also exploring more refined prompting strategies. moreover, we will study the overall impact of the proposed adaptation framework on llm interpretability and reliability in other downstream tasks, like qa and dialogue. future work could also explore frame-to-frame relations (e.g., inheritance links), which are highly relevant for reasoning tasks such as nli where entailment often depends on recognizing hierarchical or causal connections between events. in addition, future evaluations should stratify framenet qa data to examine which question types (e.g., frame definitions, frame elements, lexical units) drive the observed improvements, and extend the study across multiple large language models to assess the generalizability of frame-semantic knowledge injection beyond a single architecture.