MetaPro 2.0: Computational Metaphor Processing on the Effectiveness of Anomalous Language Modeling (2024)

Abstract: Metaphor interpretation is a difficult task in natural language understanding. The development of relevant techniques in this domain is slow, mostly because of the lack of large annotated datasets and effective pre-trained language models (PLMs) for metaphor learning. Thus, we propose a large annotated dataset and a PLM for the metaphor interpretation task. Our foundation model is based on a novel anomalous language modeling (ALM) method, which we benchmark with comparable PLM baselines on the new dataset, finding that it largely improves model performance on metaphor identification and interpretation.

heading: 1 Introduction
text: A metaphor is defined as using one (a SingleWord Expression, SWE) or several words (a MultiWord Expression, MWE) to represent a different meaning, rather than its literal meaning (Lagerwerf and Meijers, 2008). Interpreting the meanings of metaphors in contexts is particularly challenging for machines (Stowe et al., 2022). There are two important tasks in linguistic metaphor processing, i.e., metaphor identification and interpretation. The former task is normally defined as a classification task, detecting metaphors on a sentence level (Heintz et al., 2013), a word-pair level (Ge et al., 2022), or a token level (Stowe et al., 2019). The latter task is normally defined as a property extraction task (Su et al., 2020), a paraphrasing task (Mao et al., 2018), or an explanation pairing task (Mao et al., 2022). Token-level metaphor identification, and paraphrasing-based metaphor interpretation are more supportive for natural language processing (NLP) downstream tasks, because metaphors can be paraphrased into their literal counterparts without breaking the coherence and general syntax of the original sentences, e.g., “I don’t buy1 your story” vs “I don’t believe your story”.
1Italics denote metaphors.
Machines can directly use paraphrased sequences to improve metaphor understanding in downstream tasks. Steen et al. (2010) have developed the largest token-level metaphor identification dataset, VU Amsterdam Metaphor Corpus (VMC). Many previous metaphor identification models were developed upon it (Choi et al., 2021; Mao and Li, 2021; Li et al., 2023a,b), advancing the task significantly. In contrast, the progress of metaphor interpretation has been falling behind. This is likely because of the lack of a large paraphrasing-based metaphor interpretation dataset and an effective pre-trained language model (PLM) for the learning of metaphor interpretation. We are motivated to develop a metaphor interpretation-orientated dataset, VU Amsterdam Metaphor Corpus with Paraphrases (VMC-P)2, and a new PLM with a novel anomalous language modeling (ALM)3 pre-training paradigm.
We aim to develop a dataset for training end-toend (E2E) metaphor interpretation systems. We believe that a good metaphor paraphrase dataset should contain sufficient reliable training data; Metaphors in the dataset should be used in a manner similar to everyday language. Compared to the latest metaphor interpretation dataset (IMPLI) (Stowe et al., 2022) (to the best of our knowledge, it may be also the largest) which contains 920 manually paraphrased metaphors and idioms and 17,027 automatically paired paraphrases, our VMCP dataset has more manually annotated metaphor paraphrases, accounting for 11,880 units, covering both SWEs and MWEs. Besides, each sentence in IMPLI only has a paraphrase for a single target metaphor, which is dissimilar to the real-world distribution. In contrast, the sentences in our dataset can have multiple metaphors and corresponding paraphrases (2.2 metaphor units per metaphorical sentence on average).
2 https://huggingface.co/datasets/RuiMao1988/VMC-P 3 https://huggingface.co/RuiMao1988/ALM
9891
In real-world texts, it is common for a sentence to have multiple metaphors (see the example in Figure 4). Thus, our dataset is closer to real-world scenarios. There is another metaphor paraphrase dataset (Bizzoni and Lappin, 2018), while it just contains 200 gold paraphrases. Our proposed pretraining paradigm, ALM, is metaphor processingtailored and linguistics-informed, compared to current masked word prediction (MWP)-based PLMs, e.g., BERT (Devlin et al., 2019) and RoBERTa (Liu et al., 2019), language modeling-based PLMs, e.g., GPTs (Radford et al., 2018, 2019; Brown et al., 2020), and sequence-to-sequence-based PLMs, e.g., T5 (Raffel et al., 2020). ALM is motivated by the fact that metaphors do not take their literal meanings in contexts. We hypothesize that the literal meanings of metaphoric words are likely contextually anomalous. For example, given “my car drinks gasoline”, literally, neither a car can drink, nor can gasoline be drinkable. Wilks (1975, 1978) explained this as Selectional Preference Violation (SPV) – metaphors violate the selectional preference of their contexts. We hypothesize that a randomly replaced word also likely violates the selectional preference of its context. Thus, we can develop a pre-training corpus with randomly replaced words to simulate the SPV of metaphors.
ALM aims to detect anomalous replacements and retrieve original words to simulate metaphor identification and interpretation tasks, respectively. The pre-training corpus is free of human annotation as we can automatically replace an SWE/MWE with a random one, achieving a large pre-training corpus with many anomalies. We use contrastive learning to simulate a human Metaphor Identification Procedure (MIP) (see Appendix B, Pragglejaz, 2007): A metaphor is identified based on the semantic contrast between its contextual and basic meanings. Contrastive learning of positive and hard negative samples also helps ALM to learn the boundary between them, which is an important human knowledge acquisition process (NASEM, 2018).
We re-train RoBERTa-large with pre-processed WIKIPEDIA that contains hard anomalous SWEs and MWEs. We compare the re-trained model, also termed ALM, with parameter size-comparable PLMs, e.g., large BERT, RoBERTa, medium GPT2, and T5-base, finding that ALM exceeds the best PLM by 3.20% and 3.60% averaged F1 in metaphor identification and E2E interpretation tasks, based on VMC-P dataset. The fine-tuned ALM for metaphor processing is termed MetaPro 2.0.
We hope the preliminary success of our pretraining paradigm in a small PLM may inspire future large language model (LLM) development, although our model may not be able to exceed them in the same fine-tuning setups. Our contribution is twofold: (1) We propose a new dataset for training E2E metaphor interpretation systems. To our knowledge, this is the largest manually developed, token-level metaphor paraphrase dataset. (2) We propose a PLM with a novel ALM paradigm for metaphor interpretation, which outperforms previous PLMs with similar parameter sizes.

heading: 2 Related Work
text: Metaphor identification is a well-studied area, because of rich annotated data resources (Birke and Sarkar, 2006; Steen et al., 2010; Mohammad et al., 2016; Xu et al., 2022). Currently, the token-level metaphor identification task is likely formulated as a sequence tagging task (Stowe et al., 2019; Chen et al., 2021; Li et al., 2021), using PLMs, e.g., BERT and RoBERTa. Thus, an effective PLM is important for the success of metaphor detection. Metaphor interpretation is underdeveloped. Su et al. (2020) interpreted metaphors by extracting properties that were shared by source and target domains. Bizzoni and Lappin (2018) paired metaphoric sentences with handwritten paraphrases on the sentence-level. Mao et al. (2018) paraphrased metaphors on the token level without testing MWEs. Mao et al. (2022) proposed an E2E model for identifying and interpreting metaphoric SWEs and MWEs. The SWE interpretation was given by the original RoBERTa-based MWP without fine-tuning, due to the lack of labeled data. They introduced a metaphoric MWE dictionary to explain a metaphor via a clause after an original sentence. Mao et al. (2023b) integrated a concept mapping method, based on metaphor and its paraphrase (Mao et al., 2022), showing the utility of such a metaphor processing paradigm in cognitive analysis (Han et al., 2022; Mao et al., 2023a, 2024b). However, Mao et al. (2022) have some limitations including gaps between the probability distributions of the raw RoBERTa MWP and the distributions of metaphor paraphrases in contexts. Metaphor interpretation datasets were built for tasks such as definition-pairing (Zayed et al., 2020), sentence-level paraphrases (Bizzoni and Lappin, 2018), and natural language inference (NLI) testing (constitution-level) (Stowe et al.,
2022; Chakrabarty et al., 2022). The definitionpairing-based dataset includes metaphor definitions from dictionaries, which are independent of contexts. Unlike sentence-level interpretation datasets that paraphrase the meaning of a metaphoric sentence as a whole, finer-grained interpretation datasets (Stowe et al., 2022; Chakrabarty et al., 2022) paraphrase metaphoric SWEs and MWEs within contexts, and does not break up the general syntactical structures of original sentences, because the literal context words are not changed after paraphrasing. Thus, finer-grained interpretation likely yields more context-consistent paraphrases and reduces the uncertainty of the learning for machines. Since Stowe et al. (2022); Chakrabarty et al. (2022) aimed to study figurative NLI, each target sentence contains just an in-context SWE/MWE paraphrase, which does not link well with everyday language where a sentence can have multiple metaphors. Besides, these datasets include clear-cut figurative examples for NLI testing, thus, they may be suboptimal for training a metaphor processing model for paraphrasing real-world texts (Shutova, 2015). PLMs are critical for metaphor identification and interpretation. Researchers developed different pretraining tasks and architectures to achieve contextualized representations for input sequences, e.g., MWP tasks (BERT and RoBERTa), language modeling tasks (GPT family), and sequence-tosequence learning tasks (T5). These PLMs were trained for general learning purposes, whereas the pre-training task setups were not tailored by the linguistic intuition of metaphors. For example, language modeling and sequence-to-sequence learning methods did not take bi-directional contexts into account. MWP methods used a special token “[MASK]” to replace one or several original tokens to learn their representations from bi-directional contexts, whereas, MWP ignored the semantics of the original masked-out words, simply learning from their contexts. These setups are sub-optimal for the learning of metaphor processing because metaphors provide necessary semantic information in the context for their interpretation; both the forward and backward contexts help to identify metaphors and interpret their intended meanings.

heading: 3 VMC-P Dataset Development
text: We aim to develop a metaphor interpretation dataset that helps to yield E2E systems for real-world application scenarios. We define the interpretation task
as metaphoric SWE and MWE paraphrasing (tokenlevel interpretation). Our dataset contains 10,716 sequences, where 50.12% of them are non-literal. We have paraphrased 11,880 metaphor units (SWEs and MWEs). The detailed statistics and an example of our dataset can be viewed in Appendix C.
We source data from VMC, as it is the largest all-word annotated metaphor identification dataset. The VMC data were sourced from the everyday language with academic, fiction, news, and conversation genres. We focus on labeling open-class metaphors as they deliver richer semantic information. Their paraphrases will benefit more downstream tasks. If a metaphoric MWE unit includes words with other Parts-of-Speech (PoS), we paraphrase the MWE as a whole. Metaphor identification serves interpretation in our dataset. If a metaphor cannot be paraphrased within its context without breaking the precision and conciseness of language, it is not labeled as a metaphor. We cancel 8,503 (40.2%) original metaphor labels from VMC, including 6,895 (32.6%) closed-class metaphors.
Metaphor interpretation is subjective and creative (Indurkhya, 2007), which is the key annotation challenge. To improve labeling consistency, we took the following measures: First, we employed an English-speaking expert with a linguistic education background to annotate the full dataset4. Thus, the only annotator can generate consistent paraphrases for metaphors5. Second, we used WordNet (Fellbaum, 1998) hypernyms and synonyms as annotation references6. We developed a dictionary (D) that allows a user to query hypernyms and synonyms of semantic units. The annotator was asked to choose the most appropriate paraphrase lemma with the same PoS as the original metaphor from D. If a suitable option was not available, the annotator then wrote a paraphrase by his/her own understanding. The references from the dictionary help the annotator generate consistent paraphrases. Besides, using the dictionary as a knowledge base is computation-friendly.
4The Singaporean annotator and evaluators noticed the task is a part-time job for NLP research, paying $9 per hour (local part-time rate is $9: https://www.mom.gov.sg/employmentpractices/progressive-wage-model/local-qualifying-salary). They were asked to take a break every 30 minutes.
5Introducing multi-annotators may lead to increased variability in our annotation task, making it more difficult to establish definitive ground truth. Thus, we use an annotate-thenevaluate process. The rationality is discussed in Appendix A.
6When it comes to the annotation and machine learning of non-English metaphors, one can use BabelNet (Navigli and Ponzetto, 2012) as a structured resource of synonyms.

heading: 3.1 Metaphor interpretation annotation
text: The annotator was trained with MIP (see Appendix B) first. After achieving at least 90% correct metaphoric paraphrases (including correctly identifying metaphors) out of 50 instances, the annotator started to annotate the paraphrases in VMC-P.
First, the annotator would read a text from VMC, and understand the overall meaning. Then, the annotator would interpret the contextual meaning and basic meaning of a given metaphor in the text. If the contextual meaning contrasts with the basic meaning, and the contextual meaning can be understood in a comparable way, the annotator would generate a paraphrase whose basic meaning can represent the contextual meaning of the metaphor as the gold paraphrase. The generated paraphrase should be coherent with the context and use the same word form as the original metaphor. If the contextual and basic meanings are not contrastive, or there is no better paraphrase to represent the metaphor, the annotator would mute the original metaphoricity label. Such a case likely appears in dead metaphors, e.g., “falling in love”. Cognitively, the metaphoricity of “falling in” is given by the fact that the source and target concepts are from different domains, e.g., LOVE IS SPACE. However, it is hard to find a paraphrase whose basic meaning represents the contextual meaning of “falling in” and keep the language concise and precise. Paraphrasing these metaphors is sub-optimal for using metaphor processing for downstream task pre-processing, so we did not paraphrase them. We also canceled the metaphoricity of scientific words, e.g., “sausage instability” and “electric field”, because these words have been widely recognized and accepted by the public. We cannot find better paraphrases for them.
The annotator was advised to generate a singleword replacement as a paraphrase for easier computing. If a single-word replacement loses the emotional connotation of the original metaphor, an additional modifier is added, e.g., paraphrasing a verbal metaphor with an adverb-verb phrase (“Adam snarled” → “Adam angrily said”). If a metaphor is an MWE, e.g., idioms and prepositional phrases, the annotator would consider the consecutive words in the text as a whole for paraphrasing (“shaken a large fist” → “demonstrated the anger”; “filtering out” → “separating”). A simile would be paraphrased as another simile, while the paraphrase would be closer to the intended meaning of the original simile (“like a snake” → “like a bad per-
son”). The annotator was advised to sparingly generate WordNet uncovered paraphrases to maintain annotation consistency. Any newly generated paraphrases were automatically added to the reference dictionary D for later references.
After obtaining the gold paraphrase (positive sample) of a metaphor, the corresponding negative samples are also included in our dataset, which are the WordNet hypernyms and synonyms of the original metaphor that were not selected by the annotator. They are sub-optimal paraphrases for the annotator. Negative samples of WordNet uncovered SWEs or MWEs were sourced from Google by searching their synonyms. Negative samples also include annotator-generated paraphrases for other metaphors with the same lemma as the current metaphor, while the paraphrases differ from the current gold paraphrase. Thus, one can use those samples in contrastive learning and evaluate a metaphor interpretation model by measuring if the model can select the gold paraphrase from the collection of positive and negative samples. Given a PoS, if the number of negative samples is lower than three, negative samples will additionally include the hypernyms and synonyms of the metaphor from other PoS, so that we can collect sufficient hard negative samples for contrastive learning. The negative samples are hard samples as they are semantically similar to the original metaphor in different senses, and most of them have the same PoS and word forms as the original metaphor.

heading: 3.2 Quality control
text: We invited two native English speakers from Singapore to evaluate all the expert-annotated 11,880 gold paraphrases. They evaluated if a given paraphrase of a metaphor is acceptable in the context by the annotation criteria. The average accuracy voted by the two raters is 99.79%. 99.64% paraphrases were rated as “acceptable” by both raters (agreed by the three participants); 0.30% were “acceptable” for one of the raters (agreed by the two participants); 0.07% were “unacceptable” for both raters. We re-evaluated the paraphrases disagreed by both raters and corrected the corresponding errors.

heading: 4 ALM Pre-training and Fine-tuning
text: A new PLM is also proposed in this work, based on a novel ALM paradigm. ALM-based pre-training is inspired by two linguistic findings about metaphors: (1) the literal meaning of a metaphor violates the
selectional preference of its context (SPV); (2) a metaphor can be identified if there is a semantic contrast between its contextual and basic meanings (MIP). Finding 1 motivates us to develop a pretraining corpus with anomalous word replacements that violate the selectional preference of contexts. Then, a PLM learns to detect anomalies. Finding 2 motivates us to use contrastive learning to learn the semantic contrast between the real meaning of a word and its other meanings. Then, the PLM learns to retrieve the original words by pulling the representation of a replaced word close to the representation of the original word (real meaning) and pushing the representation of the replaced word distant to the representations of its other synonyms (other meanings). We use the synonym of an original word instead of [MASK] to represent the anomaly, because we assume that the synonym replacement has necessary semantic information in the context, which is similar to a metaphor. The above setups simulate metaphor identification and interpretation tasks, respectively. Thus, a PLM can learn more useful task-specific knowledge for computational metaphor processing via the ALM paradigm, as opposed to the MWP, language modeling, and sequence-to-sequence learning paradigms.
To support both metaphor identification and interpretation tasks, our pre-training employs a multi-task learning (MTL) framework to learn the anomaly detection task (a sequence tagging task corresponding to metaphor identification in finetuning) and contrastive learning task (a word classification task corresponding to metaphor interpretation in fine-tuning) together.

heading: 4.1 Pre-training corpus preparation
text: We use English WIKIPEDIA as the pre-training data source. We aim to develop a pre-training corpus that simulates the SPV of metaphors and supports the learning of semantic contrasts between the real meaning of a word and its other meanings within a context. WIKIPEDIA is used because it likely uses literal expressions to introduce scientific knowledge with formal language. It was also used by other PLMs (Devlin et al., 2019; Liu et al., 2019). Thus, aside from implementing a novel pre-training paradigm, we did not incorporate additional corpora to enhance the pre-training outcomes.
We hypothesize that the meaning of a word can be symbolically represented by one of its synonyms and hypernyms under a specific sense. For example, the verb “buy” has several senses in WordNet,
e.g., “obtaining by purchase”, “accepting as true”, etc. Given the first sense, “buy” equals “purchase”, e.g., “I will buy/purchase a new laptop”. Given the second sense, “buy” equals “believe”, e.g., “I can’t buy/believe this story”. Thus, the sense of “buy” should be closer to the sense of “believe” than “purchase” in the context of “I can’t buy your story”. Given “I can’t believe your story”, “believe” satisfies the selectional preference in the context. If we replace “believe” with another word, the word likely violates the selectional preference.
We randomly select a synonym of an original word from WordNet to replace the original word, where the replacement is defined as an anomalous word in a context7. Synonyms are mutual, e.g., “buy” is the synonym of “believe”, and vice versa. Then, in the later fine-tuning stage, the real meaning of a metaphor can also be retrieved from one of its candidates8. The original word is considered as the positive meaning of the anomalous word. Next, we sample n (a hyperparameter) synonyms of the anomalous word, excluding the original word, as negative meanings.
Our pre-training task is to detect the anomalous word and retrieve the original word (the real meaning; a positive sample) out of other synonyms of the anomalous word (other meanings; negative samples). We randomly select 35% open-class words of a sentence to replace them to develop a pretraining sequence. The criterion of selecting the original word for replacing is that the frequency of the lemma of a selected word is above the median within WIKIPEDIA so that we can collect sufficient positive samples with different contexts for contrastive learning. The anomalous words, positive and negative words can also be MWEs because WordNet contains MWEs. If there is no word satisfying the criterion, the sentence is not included in the pre-training corpus. To obtain hard negative samples and anomalous words, their word forms are aligned to the original word. The word form
7For non-English pre-training, one can also use word embeddings for semantically similar replacement acquisition.
8We include synonyms and hypernyms as candidates for VMC-P dataset development and fine-tuning, because hypernyms also have similar meanings as their covered words, although the hypernym meanings are relatively more abstract than synonyms. More candidates (using both synonyms and hypernyms) yield more annotation references and negative samples for fine-tuning. We do not include hypernyms as negative samples for pre-training, because we can achieve sufficient pre-training data with synonyms only. Furthermore, synonyms may be harder negative samples than hypernyms for more effective contrastive learning-based pre-training.
alignment method is from Mao et al. (2022). To obtain the frequency statistics and the word forms of anomalous words and negative samples, we run PoS tagging and lemmatization on full WIKIPEDIA.

heading: 4.2 ALM-based pre-training
text: Our pre-training is upon RoBERTa-large9 and Byte-Pair Encoding (BPE) tokenizer (Radford et al., 2019), due to its effectiveness in metaphor detection (Leong et al., 2020). Given a text sequence with m anomalous words (a), l original words (w), and n randomly sampled negative words (r) of awii , a pre-training input (s) is organized as s = 
w1w2...a wi i ...a wj j ...wl
r1
r2
...
 wi
...
rn
, where a wi i denotes the word or MWE at Position i is anomalous, corresponding to the original word or MWE wi. 
 and 
 are special tokens. An input sequence can have multiple anomalous words and a prompt. We define the sequence after the first 
 as a prompt (the blue input tokens in Figure 1), including a positive sample (wi) and n negative samples of awii . The position of wi is random in the prompt.
An MTL framework is used for pre-training (Figure 1). For each training step, ALM learns to detect all anomalous words via a sequence tagging task and retrieve the original word (wi) of an anomalous word (awii ) out of n + 1 candidates via a prompt and contrastive learning. Learning the next original word (wj) of the sequence is in a different training step with another prompt. awii is an anchor, pulling its representation close to the representation of wi and pushing its representation distant to the representations of negative samples via con-
9 https://huggingface.co/FacebookAI/roberta-large
trastive learning. Thus, the model can learn the semantic contrast between the real meaning and other meanings within a context. If awii , wi or r has multi-tokens, we average the hidden states (h in Figure 1) of all the tokens for contrastive learning. Upon ALM, we use a linear layer to identify all anomalous words. We use a cross-entropy loss to learn the sequence tagging task (L(seq))
L(seq) = CrossEntropy(Ŷ (seq), Y (seq)).
The contrastive learning loss (L(con)) is from He et al. (2020) with an Euclidean distance measure
L(con)=− ∑
i
log exp(E(hi, h ′ i)/τ)∑
j∈{1,...,n} exp(E(hi, h ′ j)/τ)
,
where E(·) denotes Euclidean distance. h and h′ denote the hidden states of anomalous sentences and prompts from the ALM output, respectively. τ denotes a temperature hyperparameter. The overall loss (L) is the weighted sum of the learning tasks
L = αL(seq) + βL(con),
where α and β are hyperparameters for pre-training ALM on WIKIPEDIA.
Compared to MWP-based pre-training, the advantages of ALM are summarized as follows: (1) ALM incorporates the semantics of an anomalous word (a synonym of the original word) to infer the original word, while MWP uses a unified [MASK] to predict the original word, although both methods use bi-directional contexts. (2) The contrastive learning allows ALM to take the semantics of the inferred positive and negative samples into account via an additional input prompt, while MWP considers the original word (a positive sample) as an output label. Taking the semantics of the predicted label words into account is helpful for learning MWEs because different word combinations may have similar or different meanings, e.g., “take”, “take in”, “take in charge” and “take in water”. (3) The predicted probability distributions of contrastive learning narrow down to the scope of limited candidates in a prompt, while MWP probability distributions cover the whole vocabulary. The complexity of ALM reasoning is greatly reduced.

heading: 4.3 Fine-tuning and testing
text: During the fine-tuning stage, ALM learns to identify metaphors (a sequence tagging task) and retrieve the gold paraphrase of a metaphor out of
k+1 candidates (a contrastive learning-based word classification task). Then, in Figure 1, awii is a metaphor whose gold paraphrase is wi. r is a negative sample. We select k negative samples for each fine-tuning step from the reference dictionary D. We examine single-task learning (STL) and MTL frameworks, respectively, because STL is the most popular framework for metaphor identification (Ge et al., 2023); MTL is our pre-training framework.
For STL (see Figure 2), we use two ALMs to learn metaphor identification and paraphrase tasks, separately, with separated loss functions (a crossentropy loss L(seq) for sequential metaphor identification, and a contrastive learning loss L(con) for metaphor paraphrase detection). The metaphor identification module learns a sequence without a prompt, while the metaphor paraphrase module learns a sequence with a prompt and a target metaphor. For MTL, we use ALM to learn both tasks together, which is similar to the pre-training framework and input sequence structure in Figure 1. The MTL overall loss for fine-tuning is defined as
L′ = α′L(seq) + β′L(con).
During the MTL testing stage, we first feed a text with a prompt containing k + 1 <pad> tokens for querying a metaphor identification output. The <pad> tokens are separated by 
. Next, we lemmatize an identified metaphor and sample its k + 1 candidate paraphrases from the reference dictionary D. The word forms of the selected candidates are aligned with the target metaphor by universal dependencies (Nivre et al., 2016), then fed into the model via a prompt. Since the k + 1 sampled words may be less than the total number of candidates of the target metaphor in D, we sample multiple times and compare them until finding the winner that is the most similar to the metaphor in vector space among all the candidates. The winner
is the predicted paraphrase of the target metaphor. If multiple metaphors are identified, the model predicts multiple times until each metaphor has a paraphrase. Testing the STL models is similar. The only difference is we directly feed testing data into the metaphor identification model without concatenating a prompt to query an identification output.

heading: 5 Experiments
text: Baselines. ALM has 355M parameters. We benchmark ALM to other parameter size-comparable PLMs, e.g., BERT-large-cased (340M), RoBERTalarge (355M), GPT2-medium (345M), and T5-base (220M), based on the MTL and STL frameworks in Figures 1 and 2. Baseline hyperparameters are tuned by the results of the validation set.
State-of-the-art metaphor processing models are not our direct baselines, because a) their technical novelties are irrelevant to pre-training paradigms; b) they are not foundation models for computational metaphor processing; c) our work focuses on E2E metaphor interpretation rather than metaphor identification; d) the only E2E metaphor paraphrase baseline (Mao et al., 2022) cannot paraphrase metaphoric MWEs in context and was not fine-tuned with metaphor interpretation datasets10, whereas these are the advancement of the proposed ALM and VMC-P. LLMs are also not comparable to ALM because of their huge amount of parameters, significant training, and inference costs. However, due to the absence of evaluating LLMs on E2E metaphor interpretation, we test GPT-3.5/4 on VMC-P in Appendix E. Pre-training Setups. ALM is pre-trained on 4 NVIDIA RTX A6000 GPUs for 21 days. The batch size is 512. The number (n) of negative samples is 5. We use Adam optimizer (Kingma and Ba, 2015) with a learning rate of 1e-5. The contrastive learning loss temperature (τ ) is 0.05. The pre-training loss weights (α and β) are 10 and 1, respectively. Fine-tuning Setups. ALM is fine-tuned on an NVIDIA V100 GPU. The batch size is 4. The number (k) of negative samples is 3. We use Adam optimizer with a learning rate of 1e-5. The contrastive learning loss temperature (τ ) is 0.05. The fine-tuning loss weights (α′ and β′) are 10 and 1, respectively. Fine-tuning early stops by 5 epochs, examined on the validation set. The reported results are averaged over 5 runs.
10The fine-tuned metaphor paraphrase module of Mao et al. (2022) equals to the RoBERTa-large baseline.
Label schema. We use BIO schema for metaphor identification labels. B denotes the beginning token of a BPE tokenized metaphor (SWE or MWE); I denotes inside the metaphor; O denotes a literal. We use F1 as the main measure for identification and interpretation tasks (see Appendix D for evaluation metrics and computational details).

heading: 6 Results
text: We evaluate fine-tuned ALM, termed MetaPro 2.0, with STL and MTL, based on VMC-P. More sequence tagging evaluation tasks can be viewed in Appendix F. We show the metaphor interpretation results with gold (G) and predicted (P) identification labels. Since we aim to offer useful resources for E2E metaphor interpretation, metaphor interpretation performance with predicted metaphor identification labels is the primary concern.

heading: 6.1 Overall performance
text: As shown in Table 1, MetaPro 2.0-P achieves large gains in F1 scores over the strongest baseline (RoBERTa-P) on the E2E metaphor interpretation task with predicted metaphor identification labels, under STL (+3.93%) and MTL (+3.26%) setups. This indicates the effectiveness of ALM pre-training paradigm, because we did not use addi-
tional pre-training corpora compared to RoBERTa. Compared to non-MWP-based PLMs, the improvements over GPT2 (STL: +10.09; MTL: +10.23%) and T5 (STL: 7.63%; MTL: 7.07%) are larger, indicating the utility of bi-directional context learning in our pre-training. The improvements under the E2E setup can also be attributed to the accuracy gains of MetaPro 2.0 in metaphor identification and gold identification label-based metaphor interpretation. For example, MetaPro 2.0 outperforms RoBERTa by 2.4% (STL) and 4.0% (MTL) in metaphor identification. MetaPro 2.0-G outperforms RoBERTa-G by 2.70% (STL) and 2.54% (MTL) in gold identification label-based metaphor interpretation. STL models often outperform MTL models due to the absence of sophisticated encoders and soft-parameter sharing mechanisms that optimize the learning of task-specific features in MTL. Finally, there is a huge performance gap between metaphor identification and interpretation tasks across all the models, showing that the latter task deserves more research efforts in computational metaphor processing.

heading: 6.2 Breakdown analysis
text: We compare different models by open-class breakdowns in Table 2, finding that MetaPro 2.0 exceeds the baselines across all setups on both detection and interpretation tasks. Additionally, 36.88% and 36.11% of metaphoric MWEs can be correctly paraphrased by MetaPro2.0-STL-G and MetaPro2.0MTL-G, respectively, exceeding BERT-G (STL: 28.46%; MTL: 26.21%), RoBERTa-G (STL:
33.21%; MTL: 34.63%), GPT2-G (STL: 24.64%; MTL: 24.64%), and T5-G (STL: 28.93%; MTL: 26.07%). This supports our argued advantage of ALM that taking the semantics of the predicted positive and negative samples into account is helpful for the learning of MWEs.

heading: 6.3 Hyperparameter analysis
text: As shown in Table 3, including more candidates largely reduces training and inferring time costs. However, more candidates decrease F1 for metaphor identification and interpretation somewhat, because the longer prompt may impact the semantic representations of the original sentence. More candidates in a prompt also result in lengthy inputs. The longest input sequence of MetaPro 2.0 is 512 tokens after BPE. In practice, one can balance the sequence lengths and model utilities. Using an additional light classifier to recall more likely paraphrase candidates out of the full candidate list, then retrieving the best paraphrase via the main model may reduce inference time costs.

heading: 7 Conclusion
text: We proposed a novel PLM, ALM, for computational metaphor processing. We also created a large metaphor interpretation dataset (VMC-P) focusing on token-level metaphor paraphrases. By finetuning ALM and other parameter-size comparable PLMs with VMC-P, we found that fine-tuned ALM (MetaPro 2.0) outperforms the baselines in both metaphor identification and interpretation tasks.
According to the survey of Ge et al. (2023), extensive research works in metaphor processing focused on metaphor identification. Although it is a foundation task, the advancement of relevant tasks, e.g., linguistic and conceptual metaphor understandings have more practical values in downstream applications, e.g., enhancing machine understanding of metaphorical language (Mao et al.,
2018, 2022), analyzing cognitive patterns of subjects through conceptual metaphor understanding (Han et al., 2022; Mao et al., 2023a, 2024b), and fostering language creativity (Yu and Wan, 2019; Stowe et al., 2021). Meeting the demands of these downstream applications necessitates the development of more functional metaphor processing systems beyond mere metaphor identification. Accurately paraphrasing metaphors plays a crucial role in bridging the gap between linguistic and conceptual metaphor interpretation, thereby providing an interpretable layer for text analysis (Wu et al., 2023, 2024; Cambria et al., 2024). We hope that the proposed ALM and VMC-P can inspire further research in this area and make a substantial contribution to the broader expansion of computational metaphor processing into downstream tasks.
Limitations
MetaPro 2.0 is a metaphor processing system, finetuned upon ALM and VMC-P dataset. It can identify and interpret English metaphors in a text preprocessing fashion. However, the current version does not support other languages besides English. Next, we did not consider metaphoric dependency between instances when we annotated the dataset (an instance is a short text with one or several sentences). The identification and interpretation of metaphors in one instance are independent of the metaphors from other instances. Thus, our system trained with the dataset is incapable of processing document-level dependent metaphors. Next, MetaPro 2.0 is a knowledge-based neurosymbolic system. If the paraphrase of a metaphor is out of the coverage of our developed knowledge base (the reference dictionary D), MetaPro 2.0 cannot yield an accurate interpretation for it. Finally, MetaPro 2.0 cannot directly process very long texts, e.g., more than 512 tokens after BPE (the maximum of 512 tokens includes the tokens of the concatenated prompt). In practice, we will segment very long texts first, then feed them into our system.
Ethics Statement
This article follows the ACL Code of Ethics. The annotations are based on a public dataset that does not contain private data. The tool we developed is a data preprocessing technique for improving human and machine understanding of metaphorical language. To the best of our knowledge, there are no foreseeable potential risks of using this technique.

heading: Acknowledgments
text: This research/project is supported by the Ministry of Education, Singapore under its MOE Academic Research Fund Tier 2 (STEM RIE2025 Award MOE-T2EP20123-0005) and by the RIE2025 Industry Alignment Fund – Industry Collaboration Projects (IAF-ICP) (Award I2301E0026), administered by A*STAR, as well as supported by Alibaba Group and NTU Singapore.

heading: A Rationality of the Annotation Method
text: Metaphor interpretation is a highly subjective and creative process, influenced by individuals’ intuitions and personal experiences (Indurkhya, 2007). Despite this, there is a lack of theoretical research providing practical guidance for humans to paraphrase metaphors with minimal divergence. Consequently, achieving independent metaphor paraphrase annotations with high-level agreement is challenging, as there can be multiple valid interpretations for the same metaphor. For instance, the metaphorical expression “she devoured his novels”, could be paraphrased as “she avidly enjoyed his novels” or “she eagerly read his novels”.
Considering the purpose of developing VMC-P dataset is to train E2E metaphor processing systems that can be used in real-world applications (as we argued at the beginning of Section 3), our annotation method involves an annotate-then-evaluate process. Initially, an expert annotates metaphor paraphrases, which are then evaluated by two independent raters to determine their acceptability.
This annotation process is rational for several reasons: (1) An E2E metaphor processing system trained on our VMC-P dataset is supposed to produce human-acceptable metaphor paraphrases, making the dataset unnecessary to exhaustively gather all possible paraphrases of the same metaphor from human annotators. (2) Due to the subjective nature of metaphor interpretation, obtaining majority-agreed paraphrases from independent annotations is challenging, as the selected annotators may not represent the preference of a wider population. Different annotation groups likely produce varying majority-agreed paraphrases in this domain. (3) Utilizing an expert with profound linguistic knowledge and responsibility to annotate metaphor paraphrases can ensure the annotation accuracy. A single expert annotator can provide more consistent annotations across various tasks compared to distributing the tasks among multiple annotators in a crowd-sourcing approach. Consistent annotations are particularly beneficial for machine learning models in understanding data patterns. (4) The high agreement rate among the two independent raters (see Section 3.2) can further validate the expert’s annotations, ensuring the dataset’s quality.
Therefore, the annotate-then-evaluate process enhances the utility (e.g., obtaining more humanannotated data within specific cost constraints) and reliability (scientific validation by external raters)
of the developed VMC-P dataset for training E2E metaphor processing systems.

heading: B Metaphor Identification Procedure
text: The content (Metaphor Identification Procedure, MIP) in Figure 3 is from the work of Pragglejaz (2007), which is part of the material we used to train the annotator and raters for the metaphor interpretation annotation task. It is also one of the linguistic foundations that inspired the design of our anomalous language modeling-based pretraining paradigm and the following fine-tuning. Specifically, the contextual meaning of an original word/a metaphor contrasts with the basic meaning of the anomalous lexical substitution/metaphor in pre-training/fine-tuning. Thus, (1) both anomalous words and metaphors are distinguishable from context words; (2) the contextual meanings of anomalous words/metaphors should be closer to the meanings of original words/literal counterparts of the metaphors and further from other candidates that represent different meanings to make sense of a
sentence in semantic space.
C Dataset Information
In line with VMC dataset11, our VMC-P dataset is also licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License12. An example of data in our dataset can be viewed in Figure 4. ‘ID’ is the index of an instance in our dataset. The ‘doc_ID’ and ‘sent_ID’ are inherited from the original VMC dataset. ‘sent’ is the original sentence with tokenization. ‘metaphor_index_list’ denotes the position of a metaphor. If a sublist has more than 1 element, it means the words corresponding to the indices within the sub-list are metaphoric MWEs, e.g., “play with” at Indices 7 and 8, respectively. ‘pos_list’ denotes the gold paraphrases. ‘neg_list’ denotes negative samples. The elements of ‘metaphor_index_list’, ‘pos_list’, and ‘neg_list’ are aligned by indices. ‘lemma’ and ‘pos_tags’ were generated by spaCy ‘en_core_web_sm’ (Honnibal et al., 2020), indicating the lemmatized sequence and the PoS sequence, respectively. The PoS tags follow the Universal Dependencies scheme (Nivre et al., 2016). ‘open_class’ denotes if a token at the same position is a verb, a noun, an adjective, an adverb, or others (O). In order to ensure that future evaluations can be maintained at the same statistical standard, we tokenize the original sentences and include the PoS
11 https://www.vismet.org/metcor/license.html 12 https://creativecommons.org/licenses/by-sa/3.0/
labels. ‘genre’ denotes the genre of the text, which is inherited from the original VMC dataset, including academic texts, news, fiction and conversation.
The distributions of sequence lengths can be viewed in Figure 5. The detailed statistics of our VMC-P dataset can be viewed in Table 4.

heading: D Evaluation Metrics
text: We use F1 scores as the main metric for metaphor identification and interpretation overall evaluation.
F1 = 2× precision× recall precision+ recall ,
precision = TP
TP + FP ,
recall = TP
TP + FN .
The metaphor identification overall evaluation is evaluated on the unit level. True Positive (TP) is defined as a predicted metaphoric unit whose ground truth label is also metaphoric. A metaphor unit is a metaphoric SWE or MWE. A metaphoric MWE prediction is correct only if all the B and I labels of it are correctly predicted. False Positive (FP) is defined as a predicted metaphoric unit whose ground truth label is literal. False Negative (FN) is defined as a predicted literal unit whose ground truth label is metaphoric. In metaphor identification breakdown analysis, we evaluate the performance of different open-class PoS on the token level (the tokens in ‘sent’ in the VMC-P dataset), because we did not annotate literal MWEs. We use accuracy as the breakdown evaluation metric. The accuracy of a PoS breakdown of metaphor identification is defined as the number of correct predictions on the token level above the total number of tokens within the PoS. A metaphor identification prediction is correct if the predicted label is the same as the ground truth label on the token level.
The metaphor interpretation overall evaluation is evaluated on the unit level. TP is defined as both the metaphor identification and paraphrase predictions are the same as the ground truth metaphoric identification and paraphrase labels. FP is defined as a predicted metaphoric unit whose ground truth label is literal, and a predicted metaphoric unit whose ground truth label is metaphoric, while the predicted paraphrase of the identified metaphoric unit is different from the ground truth paraphrase. FN is defined as a predicted literal unit whose ground truth label is metaphoric. In metaphor interpretation breakdown analysis, we evaluate the performance of different open-class PoS on the token level and use accuracy as the breakdown evaluation metric. We use ground truth metaphor identification labels for metaphor interpretation breakdown analysis. The accuracy of a PoS breakdown of metaphor interpretation is defined as the number of
correct paraphrased tokens above the total number of tokens that should be paraphrased within the PoS. A metaphor interpretation prediction is correct if the predicted paraphrase is the exact same as the ground truth paraphrase. If the predicted paraphrase of an MWE is correct, all words within the MWE are considered as correctly paraphrased.
Our evaluation method is conservative because we consider a prediction is correct, only if it is the exact same as its gold label. We do not use rankingbased metrics, e.g., Hits @ K, because a metaphor processing system that only yields a paraphrased sequence for an input sequence is more supportive in downstream tasks, compared to a system that yields several candidate paraphrases. Besides, we cannot evaluate the quality of other high-ranking predictions by our dataset, although a metaphor can have multi-paraphrases in real-world texts.

heading: E LLM performance on VMC-P
text: Currently, a wide range of LLM evaluations have been conducted across various domains (Mao et al., 2024a). To the best of our knowledge, the assessments focusing on metaphor identification and interpretation by LLMs have not yet been thoroughly explored. This gap exists due to the lack of extensive annotated datasets that encompass both the identification and interpretation of metaphors. Thus, we demonstrate the preliminary performance of GPT-3.5-turbo and GPT-4 (OpenAI, 2023) on our developed VMC-P testing set. While the results of GPTs and ALM are presented together in Table 5, it does not mean that the two types of models are objectively comparable for the following reasons: a) GPT-3.5-turbo and GPT-4 were not explicitly fine-tuned for computational metaphor processing tasks; b) The parameter sizes of GPT3.5-turbo (175 billion) and GPT-4 (1.76 trillion) are much larger than ALM (355 million); c) GPT3.5-turbo and GPT-4, designed for generation, and MetaPro 2.0, developed for E2E metaphor interpretation, serve distinct purposes and entail varying deployment and inference costs. The inclusion of results from these models in the same table serves the purpose of illustrating the disparity between a small expert system (MetaPro 2.0) and robust generative AI (GPT-3.5-turbo and GPT-4) in terms of their strengths and weaknesses on our newly developed VMC-P dataset.
We used OpenAI API to access GPT-3.5-turbo (model name: gpt-3.5-turbo; temperature: 0; top_p:
1) and GPT-4 (model name: gpt-4; temperature: 0; top_p: 1) on 8th February 2024. There were two prompts employed in our large GPTbased metaphor identification evaluation, namely direct prompting (GPTDP) and Chain-of-Thought prompting (GPTCOT). The direct prompt is designed as follows:
You are a linguistic expert in metaphor analysis. Given
the text below, the task is to identify metaphors from the
text. Please tokenize the textual string by whitespace
and return the identified metaphorical tokens only. If
there is no metaphor in the text, return NA. Use concise
language to present the result.
Text: [a testing instance]
The text is a testing instance from VMC-P (e.g., joining tokens in the ‘sent’ list of Figure 4 with whitespaces). In order to augment the logical reasoning capabilities of the large GPTs, we incorporate a Chain-of-Thought methodology (Wei et al., 2022) into our prompts. Recognizing the theoretical robustness of MIP (Pragglejaz, 2007) in guiding human annotation of metaphors, we concatenate the direct prompts with the content of MIP (refer to Figure 3) as the Chain-of-Thought prompt.
For evaluating large GPTs in the metaphor interpretation task, the prompt is designed as follows:
You are a linguistic expert in metaphor analysis. Given
the text below and a candidate list, which one is the best
literal counterpart for the metaphor “[a metaphor]” among
the candidate list? Only return a choice with concise
language in the answer. Do not explain why.
Text: [a testing instance]
Candidates: [the combination of a gold paraphrase and the
corresponding negative samples]
The text is also a testing instance from the proposed VMC-P dataset. Each evaluation iteration only focuses on a target metaphor (a gold identification label-based, or predicted identification label-based metaphor). The candidates are the combination of the gold paraphrase of the target metaphor and its corresponding negative samples. The position of the gold paraphrase is randomized among the candidates. There are also two evaluation tasks, namely gold metaphor identification label-based, and predicted metaphor identification label-based paraphrase evaluation. The metaphor identification predictions are from the former prompting models, e.g., GPTDP and GPTCOT.
In Table 5, we can observe that both GPT-3.5turbo and GPT-4 yield very weak performance in
the metaphor identification task under the direct prompting and Chain-of-Thought prompting setups. This is because of their very frequent false positive errors. The large GPTs encounter difficulty in identifying fine-grained metaphorical words, because many large GPT-identified metaphors are long phrases. Such coarse-grained metaphor identification predictions do not benefit conceptual metaphor interpretation, because it would be difficult and ambiguous to abstract target and source concepts from long phrases with complicated syntactic structures (see Conceptual Metaphor Theory (Lakoff and Johnson, 1980) for the relevant concepts and examples). On the other hand, the GPTs also exhibit shortcomings in identifying metaphors, such as false negative errors, which underscores their limited proficiency in professional linguistic analysis of metaphorical language.
With MIP-informed Chain-of-Thought prompting, GPT-4COT yields significant improvements in the evaluation metrics, compared to other large GPT models. This suggests that professional linguistic knowledge, e.g., MIP, can enhance the performance of GPT-4 in accurately predicting metaphor identification tasks, although this improvement is not observed in the earlier version, GPT-3.5-turbo. This phenomenon raises a practical question regarding the use of prompt-based LLMs for computational linguistics tasks resembling metaphor identification: How can the effectiveness of prompts, originally devised for earlier or limited LLMs, be preserved across different or future versions? Examples where prompts succeeded in earlier versions of LLMs but failed in later versions can also be observed in financial sentiment
analysis tasks (Du et al., 2024). Determining a robust and effective prompt for a specific task is challenging due to the heuristic nature of many current approaches to prompt engineering (Mao et al., 2023c) and the unpredictable preferences of fast-evolved LLMs in the future. The prompting approaches, which include considerations such as phrasing, linguistic structures, as well as prompting knowledge and logic, lack a systematic methodology for identifying the optimal and robust prompt for different foundation models.
Finally, GPT-3.5/4-G outperforms MetaPro 2.0- G in interpreting metaphors when utilizing gold metaphor identification labels. This outcome is unsurprising due to the extensive pre-training data and larger parameter size of GPT-4. However, in pursuit of achieving E2E metaphor interpretation (the main goal of this work), GPT-3.5/4-PDP and GPT3.5/4-PCOT lag behind MetaPro 2.0-P in metaphor interpretation tasks, based on predicted metaphor identification labels. Considering the costs of employing large GPTs and the challenges in parsing structured data from their generated text13, MetaPro 2.0 retains advantages in E2E metaphor interpretation tasks in real-world application scenarios. The E2E processing gaps between the GPTs and MetaPro 2.0 also highlight the value of VMC-P for training expert systems in the domain of computational metaphor processing, because the relevant linguistic knowledge can not be easily learned from general corpora by pre-training.
13Although we have instructed GPT-3.5 and GPT-4 to produce only the required results, it is inevitable that the outputs may exhibit varied linguistic structures and include extraneous contextual information. Thus, it is difficult to parse the desired predictions, e.g., metaphor tokens, and the interpretation of the specific metaphor tokens from GPT-3.5/4 generated text.

heading: F Testing ALM on Other Tasks
text: We aim to propose useful resources, e.g., a dataset and a PLM for the learning of E2E metaphor interpretation in this work. However, since sequence tagging (e.g., detecting anomalous words from a sequence) is one of the pre-training tasks of ALM, we also test its performance on other sequence tagging tasks such as aspect extraction, opinion extraction, aspect-based sentiment analysis, and named entity recognition. The first three tasks are evaluated with the restaurant dataset from Pontiki et al. (2014), following the training, validation, and testing set splits of Chen and Qian (2020). Named entity recognition is evaluated with the CoNLL 2003 NER dataset from Tjong Kim Sang and De Meulder (2003), following the splits of huggingface datasets14. The label set of each task can be viewed in Table 6. As seen in the table, different tasks have different label sets and annotation paradigms, indicating the fact that the learning of those tasks needs to model different dependency relationships between words within a textual sequence.
We test the same baselines that are introduced in Section 5 on the sequence tagging tasks. As seen in Table 7, our metaphor processing-tailored PLM also has advantages in processing those nonmetaphor tasks because ALM exceeds them across all the tasks. It shows that a PLM can also gain gen-
14 https://https://huggingface.co/datasets/conll2003
eral knowledge by the ALM pre-training paradigm. While the gains in non-metaphor tasks (+1.09% F1 on average) are not as substantial as those observed in sequential metaphor identification (+3.20% F1 on average), it is important to consider the holistic utility of our PLM – excelling in metaphor processing while maintaining competitive performance in other tasks. Besides, it also shows that a task-oriented PLM can achieve better pre-training utilities than general PLMs on specific tasks. Probably, the “divide-and-conquer” strategy, e.g., using different pre-training paradigms to handle very different reasoning tasks, may be also applicable to LLMs in enhancing their performance in different domains (Cambria et al., 2023).

heading: Future Work
text: This phenomenon raises a practical question regarding the use of prompt-based LLMs for computational linguistics tasks resembling metaphor identification: How can the effectiveness of prompts, originally devised for earlier or limited LLMs, be preserved across different or future versions? Examples where prompts succeeded in earlier versions of LLMs but failed in later versions can also be observed in financial sentiment
analysis tasks (Du et al. , 2024). Determining a robust and effective prompt for a specific task is challenging due to the heuristic nature of many current approaches to prompt engineering (Mao et al. , 2023c) and the unpredictable preferences of fast-evolved LLMs in the future. The prompting approaches, which include considerations such as phrasing, linguistic structures, as well as prompting knowledge and logic, lack a systematic methodology for identifying the optimal and robust prompt for different foundation models. Finally, GPT-3. 5/4-G outperforms MetaPro 2. 0- G in interpreting metaphors when utilizing gold metaphor identification labels. This outcome is unsurprising due to the extensive pre-training data and larger parameter size of GPT-4. However, in pursuit of achieving E2E metaphor interpretation (the main goal of this work), GPT-3. 5/4-PDP and GPT3. 5/4-PCOT lag behind MetaPro 2. 0-P in metaphor interpretation tasks, based on predicted metaphor identification labels. Considering the costs of employing large GPTs and the challenges in parsing structured data from their generated text13, MetaPro 2. 0 retains advantages in E2E metaphor interpretation tasks in real-world application scenarios. The E2E processing gaps between the GPTs and MetaPro 2. 0 also highlight the value of VMC-P for training expert systems in the domain of computational metaphor processing, because the relevant linguistic knowledge can not be easily learned from general corpora by pre-training. 13Although we have instructed GPT-3. 5 and GPT-4 to produce only the required results, it is inevitable that the outputs may exhibit varied linguistic structures and include extraneous contextual information. Thus, it is difficult to parse the desired predictions, e. g. , metaphor tokens, and the interpretation of the specific metaphor tokens from GPT-3. 5/4 generated text