A Theory Guided Scaffolding Instruction Framework for LLM -Enabled Metaphor Reasoning (2024)

Abstract: Metaphor detection is a challenging task in figurative language processing, which aims to distinguish between metaphorical and literal expressions in text. Existing methods tackle metaphor detection via training or fine-tuning discriminative models on labeled data. However, these approaches struggle to explain the underlying reasoning process behind the metaphorical/literal judgment. Recently, large language models (LLMs) have shown promise in language reasoning tasks. Although promising, LLM-based methods for metaphor detection and reasoning are still faced with the challenging issue of bringing the explainable concepts for metaphor reasoning and their linguistic manifestation. To fill this gap, we propose a novel Theory guided Scaffolding Instruction (TSI) framework that instructs an LLM to infer the underlying reasoning process of metaphor detection guided by metaphor theories for the first time. Our work is inspired by a pedagogical strategy called scaffolding instruction, which encourages educators to provide questioning and support as scaffolding so as to assist learners in constructing the understanding of pedagogical goals step by step. We first construct a metaphor knowledge graph grounded in metaphor theory which serves as the instructional structure to obtain a series of scaffolding questions, directing the LLM to incrementally generate the reasoning process for metaphor understanding through dialogue interactions. During this theory guided instruction process, we explore the LLM’s mastery boundary and provide the relevant knowledge as scaffolding support when the question is beyond the LLM’s capability. Experimental results verify that our method significantly outperforms both the LLM-based reasoning methods and the SOTA methods in metaphor detection, indicating the facilitation of metaphor and instruction theories in guiding LLM-based reasoning process.

## introduction metaphor is not just a figurative expression but a pervasive phenomenon in human thought, perception, and reasoning @xcite . merriam-webster dictionary defines metaphor as "a figure of speech in which a word or phrase literally denoting one kind of object or idea is used in place of another to suggest a likeness or analogy between them". metaphor detection as a fundamental research task in natural language processing can benefit a variety of other tasks, which require the understanding of implicit semantics, such as ma-chine translation @xcite , sentiment and emotion analysis @xcite , and conversational dialogue @xcite . existing studies on metaphor detection establish supervised learning methods based on labeled data @xcite @xcite . recently, some studies employ metaphor theories to help design models for metaphor detection, resulting in significant performance gains @xcite @xcite . although these methods are inspired by different metaphor theories, they essentially train or fine-tune discriminative models to learn a decision boundary between metaphorical and literal samples, lacking the ability to explain the underlying reasoning process of metaphorical/literal judgment. recently, large language models (llms) have shown promise in generating reasoning processes with natural language across various tasks @xcite @xcite , which demonstrates their potential to provide the underlying reasoning process for metaphor understanding. although promising, llm-based methods for metaphor detection and reasoning are still faced with the challenging issue of bringing the explainable concepts for metaphor reasoning and their linguistic manifestation. figure 1 gives examples of llm-based metaphor detection methods. figure 1 (a ) and (b) show that llms fail to detect the metaphor; even when employing zeroshot chain-of-thought method @xcite . this limitation may be attributed to the fact that metaphors are not merely linguistic phenomena but associated with human thought and reasoning process @xcite . thus metaphors inherently convey intricate and implicit meanings that require a deeper level of understanding, making it more difficult for llms to comprehend. to better understand and detect metaphors, metaphor theories that provide well-established frameworks can serve as shortcuts to help effective metaphor reasoning and understanding. figure 1 (c ) provides an llm with the prompt incorporated with the key ideas from conceptual metaphor theory (cmt) @xcite , the key metaphor theory in cognitive linguistics. the result shows that, with the help of cmt theory, the llm can provide a correct answer with an explanation of corresponding reasoning process. in this paper, we tackle the challenging issue of llm-based metaphor reasoning. to better release the capability of llms for metaphor reasoning and understanding, we make the first attempt to incorporate well-founded metaphor theories in such process. further, to bridge the abstract conceptual representations of metaphor theories and the detailed llm reasoning process, we draw inspiration from a pedagogical strategy called scaffolding instruction @xcite , which emphasizes that an educator continually explores a learner's mastery boundary through dialogues and designs questions and support as scaffolding, to assist learners in constructing the understanding of pedagogical goals step by step. to achieve this, we propose a novel theory guided scaffolding instruction framework (tsi) for llmenabled metaphor reasoning. specifically, we rely on three representative metaphor theories, selection preference violation (spv) @xcite , metaphor identification procedure (mip) (pragglejaz group, 2007), and conceptual metaphor theory (cmt) @xcite . in our computational construct, we first formalize a metaphor theory by converting it into a metaphor knowledge graph (kg), which functions as the instructional structure to construct scaffolding questions that help llm's metaphor reasoning process. during dialogue interactions, our framework automatically detects the mastery boundary of llm and provides relevant knowledge as scaffolding support when the question is beyond the llm's capability. our main contributions are as follows: ## related work metaphor theories as a fundamental and profound figurative phenomenon in linguistics and cognition, metaphor has been theorized across various disciplines in linguistics, philosophy and psychology @xcite @xcite . however, not all of these theories are appropriate for computational formalization. among them, three theories are favorable in existing computational research on metaphor, including selectional preference violation (spv) @xcite , metaphor identification procedure (mip) (pragglejaz group, 2007), and conceptual metaphor theory (cmt) @xcite . the spv theory suggests that the disparity between the context of a word within a sentence and its frequently used contexts is an indicator of this word's metaphorical usage. the mip theory aims to standardize the metaphor annotation process, emphasizing that a metaphor is identified if the contextual meaning of the word differs from its basic meaning. unlike spv and mip, which utilize indicative clues based on linguistic features of contexts or word meanings to identify metaphors, cmt goes beyond linguistic analysis and proposes a cognitive basis for metaphor understanding, suggesting that metaphor facilitates a mapping of attributes from source domain to target domain in human cognition. in this paper, we focus on releasing the metaphor reasoning capability of llms with the guidance of these three metaphor theories. ## problem definition formally, d @xmath0, w k , l k )} nte @xmath1, where s k is a sentence, w k is a word within s k , and l k is the label (metaphorical or literal) for w k . our goal of metaphor detection with llm is to predict the label of the word within a sentence in d te without any training data. ## method we propose a novel theory guided scaffolding instruction framework (tsi) for llm-enabled metaphor reasoning. under the guidance of a metaphor theory, our framework instructs llm to explicitly give a reasoning process with questioning and support as scaffolding to determine whether the word within a sentence is metaphorical or literal. figure 2 shows the overall architecture of our framework, which contains four primary components: (1) metaphor knowledge graph construction, which represents a simplified metaphor theory and computational aspects in a graphical format; (2) scaffolding question construction, which constructs a sequence of scaffolding questions for an llm with metaphor knowledge graph as the instructional reference; (3) scaffolding support based on mastery level of llm, which automatically detects the current capability of llm (i.e. mastery level) and provides knowledge as support if necessary; and (4) classification, which categories the word within the sentence as metaphorical or literal, based on the comparison between the structure of knowledge graph constructed from llm's reasoning processes and the structure of metaphor knowledge graph. ## conclusion in this paper, we propose a theory guided scaffolding instruction framework for llm-enabled metaphor reasoning. inspired by a pedagogical strategy scaffolding instruction, our framework instructs an llm to infer the metaphor reasoning process using questioning and support as scaffolding guided by a metaphor theory. experimental results show that our framework outperforms the previous llm-based complex reasoning methods and the sota methods for metaphor detection, verifying the effectiveness of our proposed framework and indicating the facilitation of metaphor and instruction theories in llm-enabled reasoning.