Enhancing Language Models with Idiomatic Reasoning

Authors: jianing  zhou
ziheng  zeng
suma  bhat
Venue: colmweb_org_COLM_2024_Conference
Published: 2024-03-23 05:06:04

Abstract: Advancements in Large Language Models (LLMs) have significantly propelled the field of Natural Language Processing (NLP); however, nuanced reasoning in the presence of non-canonical language forms, such as figurative language, remains an intricate challenge. These language forms, integral to human communication, elude standard LLM comprehension due to their inherent non-compositionality, contextual ambiguity, and sparse representation in text corpora. Addressing these challenges, this paper introduces an innovative approach to seamlessly incorporate idiomatic knowledge into pre-trained language models (PTLMs). Our methodology first employs a multi-view data augmentation strategy that uses idiomatic instances representing one property to generate training data for various idiom-related tasks. When combined with a novel parameter-efficient tuning mechanism that accounts for the unique attributes of idiomatic language, we embed task-specific and idiomaticity-aware inductive biases within a PTLM.  Integrating a meta-pretraining protocol based on meta-learning principles, further equips the model with enhanced adaptability to diverse downstream idiom-aware tasks. Empirical validation on diverse benchmarks centered around idiom comprehension and reasoning, demonstrates the efficacy of our approach. Notably, our model surpasses various parameter-efficient fine-tuning baselines outperforming the conventional full fine-tuning paradigms, thereby creating more contextually aware and linguistically robust language models.

Main paper: 
 introduction: 
 Idiomatic expressions (IEs), commonly found in natural language, encompass a broad range of dynamic figures of speech that enhance fluency and concision across various genres and play a key role in human communication (Moon et al., 1998; Baldwin & Kim, 2010a) . However, pre-trained language models (PTLMs), such as BART (Lewis et al., 2020) , T5-large (Raffel et al., 2020) , and T0, struggle to reason in the presence of IEs (Chakrabarty et al., 2021b) , failing to understand their figurative semantics (Bhargava & Ng, 2022; Zeng & Bhat, 2022) . Despite their impressive scale of pre-training, even large language models (LLMs) such as GPT-4 and LlaMa2 struggle to reason effectively in the presence of figurative expressions (Liu et al., 2023; Kabra et al., 2023) . This is in part due to IEs' long tail distribution in natural language, their non-compositionality, i.e., the overall meaning of an IE is not directly derived from its individual components (Sag et al., 2002) , and their contextual ambiguity, i.e., an IE can be interpreted idiomatically or literally depending on the context (Baldwin & Kim, 2010b) . Beyond this, it is known that comprehension of idioms requires a broad linguistic awareness of the relationship between meaning and surface form, apart from commonsense associated with their use (Cacciari & Levorato, 1989) . Therefore, it is necessary to provide idiomatic knowledge to the PTLMs for better reasoning in the presence of idioms.Recent works ( Škvorc et al., 2022; Zeng & Bhat, 2021) have enhanced PTLMs for specific IE tasks, such as IE identification and usage recognition, without aiming for generalizable IE comprehension ability. Others (Zeng & Bhat, 2022; 2023) adapt PTLMs to learn semantically meaningful IE representation, thus rendering PTLMs suitable for multiple downstream IE tasks with limited reasoning abilities. A recurring obstacle encountered by previous studies is incorporating IE-related knowledge when IE-related data is scarce: MAGPIE (Haagsma et al., 2020) , the largest and the most popular IE corpus, contains 56k sentences covering 1755 IEs. As we will show in this study, scaling up pre-trained models with more parameters and larger pre-training data than ever before fails to automatically resolve this deficiency. While open-source LLMs also lack IE reasoning ability, in this study, we show how state-of-the-art IE comprehension and reasoning abilities are achievable by explicitly tackling the data scarcity challenge. Remarkably, we can accomplish this without relying on astronomically expensive pretraining data or parameter up-scaling; instead we adapt smaller-frame PTLMs to various downstream tasks through meta-learning via novel data augmentation and parameter-efficient tuning methods.Figure 1 summarizes our approach. Given a dataset of sentences with IEs, we propose multi-view data augmentation where a single dataset is transformed into an array of instances for multiple tasks, each representing an IE-related characteristic, e.g. identification task for learning contextual ambiguity, definition generation task for learning non-compositionality and generation cloze task for learning the usages. Then, we continued pre-train a PTLM on these tasks to learn generic IE knowledge. Unlike traditional data augmentation methods that generate more data for the same task or generate different data for different tasks that are irrelevant, we use the same data but transform them into different tasks. Sharing the same IEs and the different forms of sentences (as different tasks), permits the model to learn IE-related properties introduced by each task and the shared IE knowledge across tasks.To address the training efficiency challenge, we utilize parameter-efficient fine-tuning (PEFT) during continued pre-training and fine-tuning for downstream tasks. Traditional PEFT methods (Rebuffi et al., 2017; Guo et al., 2021; Hu et al.; Pfeiffer et al., 2020a) are primarily designed to learn from single-task data while ignoring linguistic processes (e.g., non-compositional language). In contrast, our data augmentation leads to mixed-task training batches containing exclusively non-compositional language. Therefore, inspired by Adapters (Pfeiffer et al., 2020b) , we propose the Idapter. It enables appropriate token representation by generating a token-level non-compositionality scalar to linearly combine the compositional and non-compositional representations in accordance with the its contextual semantics. Further, it enables a mixed-task batch training by generating a task-specific scalar to alter the intermediate representations for each task. Finally, towards orienting the model to different downstream tasks, we also propose a meta-learning based meta-pretraining mechanism that optimally initializes the model parameters for effective downstream use.To showcase the efficacy of our method, we first continue pre-training the Idapter attached to a frozen backbone PTLM using our multi-view augmented data and meta-pretraining mechanism. Then, we deploy our IE-knowledge-enhanced model on different downstream IE-related tasks (see Figure 1 ). Empirical results show our method outperforms not only other LMs that are fine-tuned fully or in a parameter-efficient manner but also open-source LLMs such as LlaMa2 and Mistral that are 200 times larger than the models we study.The contributions of this work are as follows.(1) We propose a framework that equips PTLMs with idiomatic reasoning, which could then be used to solve different downstream IE-aware tasks;(2) We address IE data scarcity via a multi-view data augmentation strategy that transforms a given dataset into a set of datasets for continued pre-training;(3) To inject idiomatic intelligence efficiently, we propose Idapter-an idiom-aware PEFT method designed for continued pre-training with multi-view data augmentation;(4) To adapt the model to different IE-aware tasks effectively, we propose our metapretraining mechanism based on meta-learning;(5) We show the efficacy of our method on a diverse set of downstream IE-related tasks. Our IE-knowledge enhanced model achieves new state-of-the-art results on IE-related tasks. 
related prior work: 
 Knowledge Injection. Recent studies have shown the promise of integrating explicit external knowledge for transformative impacts on downstream NLP tasks by continued pre-training (Zhang et al., 2019b; Beltagy et al., 2019; Lee et al., 2020; Zhang et al., 2022; Yu et al., 2022) . Among them, some works explore to inject idiomatic knowledge into PTLMs via task-specific data augmentation (e.g., generating sentence pairs for IE paraphrasing (Zhou et al., 2021c) ), leveraging external knowledge (Zeng & Bhat, 2022) , and learning with parameter efficient methods (Zeng & Bhat, 2023 ) (e.g., Adapters (Pfeiffer et al., 2020a) ). However, prior works have limited utility in that they only focus on a single task (e.g., identifying IEs (Flor & Klebanov, 2018; Amin et al., 2021; Zeng & Bhat, 2021) , disambiguating between their figurative/literal use (Liu & Hwa, 2017; 2018) or paraphrasing (Zhou et al., 2021a; b) ). Although Zeng & Bhat (2023) show their model's performance on different downstream tasks, their method is still limited since they utilize adapters without accommodating IE-specific characteristics. Besides, their training method does not mitigate the discrepancy between the training objectives and downstream tasks, rendering their method less adaptable to tackle practical downstream tasks. Additionally, none of the prior works show performance improvements in natural language understanding tasks in the presence of IEs; their performance improvements focus solely on IE processing tasks.Data Augmentation. Data augmentation, generating synthetic task-specific data, is a common, well-studied approach in the NLP community to mitigate data scarcity issues (Feng et al., 2021) . The augmented data can be derived from labeled data and directly utilized in supervised learning (Wei & Zou, 2019) or in semi-supervised learning for unlabeled data (Xie et al., 2020) . While several approaches have been proposed to address learning with limited labeled data, such as unsupervised pre-training (Devlin et al., 2019; Raffel et al., 2020) , multi-task learning (Liu et al., 2017; Augenstein et al., 2018) and semi-supervised learning (Miyato et al.; Xie et al., 2020) , these methods have limitations in IEs' domain due to IEs' non-compositionality, which complicates the generation of new texts. Consequently, we introduce a novel multi-view data augmentation technique that circumvents the need to generate new texts with IEs by transforming the original texts into different tasks.Parameter-Efficient Fine-tuning. While conventional fine-tuning updates all parameters of PTLMs, recent research has shown that updating or incorporating a minute fraction (e.g., 0.01%) of the parameters can achieve a performance comparable to that of full fine-tuning.Beginning with Adapters (Rebuffi et al., 2017) , a variety of advanced Parameter Efficient Fine-Tuning (PEFT) strategies have been developed, including selecting a sparse subset of parameters for training (Guo et al., 2021) , low-rank updates (Hu et al.) , implementing optimization in a lesser-dimensional subspace (Aghajanyan et al., 2021) , incorporate lowrank Adapters using hypercomplex multiplication (Karimi Mahabadi et al., 2021) . However, all prior PEFT strategies are designed for the more generic, compositional expressions, without any special consideration for non-compositional figures of speech such as IEs.More critically, we hypothesize that considering the figurative meanings and degrees ofInput: What is the idiomatic expression in the sentence: It looks like the temperature is going to drop tonight , so be careful not to catch a cold . Output: catch a coldInput: Fill in the mask with an idiomatic expression in the sentence: It looks like the temperature is going to drop tonight , so be careful not to [MASK] . Output: catch a coldInput: What is the definition of the idiom 'catch a cold' in the sentence: It looks like the temperature is going to drop tonight , so be careful not to catch a cold . Output: become infected with a cold Table 1 : Examples of input sentences and output sentences in our tasks. Non-compositional expressions are highlighted in bold red non-compositionality of IEs to be unique to each IE, learning each IE can be considered a separate subtask. This renders prior PEFT methods impractical for mixed-task training batches produced by our multi-view data augmentation since these methods require isolated modules for distinct subtasks. Thus, we propose Idapter with non-compositionality and task-specific scaling to aid idiomatic knowledge injection.The concept of meta-learning, also known as "learning to learn," has garnered substantial attention, resulting in a vast body of literature. Among the various meta-learning frameworks, Model-Agnostic Meta-Learning (MAML) stands out Finn et al. (2017) . Notably, MAML-related techniques have found applications across a range of natural language processing (NLP) tasks (Dou et al., 2019; Kang et al., 2020; Qian et al., 2021; Huang et al., 2023) . Despite these advancements, a critical gap remains: the application of meta-learning to pre-training of language models. While some NLP researchers have explored meta-learning (Ke et al., 2021; Wu et al., 2022) , few have delved into its potential for domain-specific knowledge injection. Our work aims to address this gap by investigating how meta-learning can enhance the pre-training of domain-specific language models, filling a niche that has yet to be fully explored in the literature. 
multi-view data augmentation: 
 Traditionally, PTLMs rely on large-scale corpora as implicit sources of domain-specific knowledge (Lee et al., 2020) . However, the long tail nature of idioms and the unavailability of comparable sources focusing on idiomatic expressions-the largest-to-date collection of sentences with IEs, MAGPIE, is made of only 56k sentences of non-uniform quality of context spanning 1,755 IEs-with sufficient contexts from which to learn their meanings, calls for novel approaches to idiomatic knowledge injection.We propose a multi-view data augmentation technique in which an available dataset of idiomatic sentences, such as MAGPIE, is augmented via different tasks derived from this dataset, to provide a diverse range of idiomatic knowledge. Specifically, given the MAGPIE sentences as input, we create datasets for different tasks by tailoring the corresponding outputs to these ends. To inform the model of IE non-compositionality and the associated figurative meaning, we create the definition generation task, in which the model is asked to produce the dictionary definition of the IE from the input. To impart IEs' contextual ambiguity, we introduce the idiom identification task in which the model is expected to extract the IEs that are used in their idiomatic sense (e.g., People talk about Alice behind her back.), while ignoring IEs that are used in their literal sense (e.g., Alice put her bag behind her back.), outputting an empty string. Finally, to allow the model to recognize proper IE uses in a given context, we add the idiom generation cloze task in which, given the contextual cues, the model is expected to fill in a mask using the IE with an appropriate figurative meaning. We resort to these tasks because, despite the pretraining objective of using the context to predict a given token/word, the emerging understanding is that this is insufficient to learn to represent idioms and much less to reason in their presence (Miletić & Walde, 2024) .As a result, for each sentence in the dataset that uses the IE idiomatically, we create three expected outputs corresponding to these three tasks. A sentence where the IE is used literally gets used only for the idiom identification task.As an illustration, the input/output pairs of these three tasks for an input sentence with the IE catch a cold are shown in Table 1 . Since all outputs are generated from the same original IE sentence, we hypothesize that by learning these tasks jointly, the model effectively learns comprehensive knowledge about the (common) IE, catch a cold, shared across tasks. In contrast to the traditional data augmentation methods that aim to generate more data for a single task and the traditional multitask learning methods that utilize different data for different tasks, our method creates different tasks from the same data, thus allowing a more effective data utilization in a low-resource scenario. 
parameter-efficient training for idiomatic knowledge: 
 Advancements in language models have significantly scaled up the number of parameters and the pretraining corpus. However, injecting new knowledge into large PTLMs using PEFT methods, rather than full fine-tuning, makes for a more efficient model. In this paper, we present an improved PEFT method that, instead of being task/process agnostic, incorporates considerations for non-compositional language, thus enabling the training of multiple tasks simultaneously. Specifically, building upon the Adapter framework, we propose an Idapter, a novel PEFT method designed to inject idiomatic knowledge into PTLMs using the multi-view augmented data.Adapters. An Adapter is a lightweight neural network placed between or inside the intermediate transformer layers of a PTLM. During training, only the Adapter weights are trainable, while the PTLM's parameters are frozen to improve efficiency and reduce training costs. Similarly, we use the paradigm where the Adapters are placed inside the intermediate layers. With our focus on idiomatic knowledge injection, the Adapter is expected to learn a different representation h a i for each token i to incorporate the potential non-compositional meaning given the original representation h i ∈ R d from the frozen PTLM:h a i = Adapter(h i )Non-compositionality Scaling. The Idapter takes into account the non-compositionality in its architecture. It has a non-compositionality scaling that generates a non-compositionality score s i n for token i:s i n = W n h i + b nwhere W n ∈ R 1×d and b n are the trainable parameters of the non-compositionality scaling layer (n stands for "non-compositionality"). Then, for each token, the final representation is the linear combination of the Adapter representation, h a i , scaled by the non-compositionality score, s n , and the original PTLM representation scaled by 1 -s n : ĥi = (1 -s i n )h i + s i n h a iwhere i refers to token i.We include the idiom identification task in our continued pre-training to teach the model about idioms' contextual ambiguity and thus disambiguate each token as part of a compositional or non-compositional phrase. Therefore, learning the idiom identification task optimizes the parameters of the non-compositionality scaling layer (referred to as NC layer).Task-specific Scaling. Because our data augmentation method formats data into different tasks, mixed-task batches are used during training, which is a second point of departure from traditional PEFT methods. The Idapter aids mixed-task batch training by learning a scalar vector for each task to help the model distinguish and optimize each task separately. Each task-specific vector is generated by a dedicated, task-specific hypernetwork used only for the inputs from its corresponding task. We consider simple, efficient linear layers as hypernetworks that are functions of the original PTLM's representations, h t ∈ R d , where t denotes a specific task in our knowledge injection phase. We implement these hypernetworks after each transformer layer from the PTLM. The hypernetwork H(•) that generates the task-specific vector l t ∈ R d is defined as:l t = H(h t ) = W H h twhere W H ∈ R d×d are the trainable parameters of the hypernetwork. The task-specific vectors are then multiplied by the sequence of representations obtained after the noncompositionality scaling as:h * i = l t ⊙ ĥi + ĥi where ⊙ represents element-wise multiplication.In effect, inputs from different tasks will be represented differently by Idapter using taskspecific scaling (referred as TS layer), making learning mixed-task batches possible. 
meta-pretraining: 
 We argue that most PTLMs' objective is to maximize their performance on pretraining tasks without considering any aspect of the downstream tasks, which results in a discrepancy between pre-trained models and downstream tasks. To alleviate this, we leverage a metalearning algorithm for pre-training optimization of the Idapter. With different augmented tasks as meta-tasks, we utilize meta-learning to update the initial parameters of our model, which could anticipate to improve downstream performance on tasks involving idiomatic expressions. Our meta-pretraining can be divided into two stages: i) meta-train stage, which updates the model's parameters θ using the train set of pre-train data:θ k = θ k-1 -α∇ θ k-1 L θ k-1 ; D train T,kii) meta-test stage, which updates the model's initial parameters θ 0 by one gradient descent step over the test set of pre-train data: where α refers to the learning rate during the meta-train stage and β refers to the learning rate during the meta-test stage. D train T,k represents the kth batch in train set of pre-train data from task T. D test T represents the test set of pre-train data from task T. The returned meta-parameters θ 0 are used as the pre-trained model parameters for different downstream tasks. We utilize Reptile (Nichol et al., 2018) , a variant of MAML (Finn et al., 2017) in our meta-pretraining stage for better efficiency.θ ′ 0 = θ 0 -β∇ θ 0 L θ k ; D 
experiments: 
 Continued Pre-training Tasks. The continued pre-training tasks aim to inject fundational IE knowledge into the models. We use the MAGPIE (Haagsma et al., 2020) for our multiview data augmentation. The original dataset was intended for idiom identification, where the task is to decide if an IE is used figuratively or literally in a given context. Thus, it serves as the data foundation for the first task of idiom identification. For the task of idiom definition generation, we retrieve the definitions from online resources, such as Wiktionary and Google Dictionary. For the task of idiom generation, we replace the whole IE with a single [MASK] token. The model is expected to reconstruct the original IE to replace the [MASK] token. By construction and the nature of the tasks, these tasks are learned via self-supervision (idiom generation) and provided labels (definition generation and idiom identification). Downstream Tasks. We further verify the effectiveness of our method on a diverse set of downstream tasks including IE-related natural language inference (IMPLI) (Stowe et al., 2022) , Idiomatic Expression Paraphrasing (I2L) (Zhou et al., 2021c) , Literal Expression Stylizing (L2I) (Zhou et al., 2021c) and Figurative Narrative Benchmark (FNB) (Chakrabarty et al., 2021a) . The tasks differ from the continued pre-training tasks described above and involve a relatively deeper interpretation of idioms. Details are provided in the appendix.Experimental Procedure. Our experiments are performed as follows: We use the T5-large model (Raffel et al., 2020) as the backbone PTLM since its multi-task pre-training aligns with our continued pre-training and thus provides better and faster adaptation. We first add an Idapter to the backbone model and then update only the Idapter during the metapretraining stage. After meta-pretraining, we deploy the model on different downstream tasks for fine-tuning. (Liu et al., 2022) . We also compare our model with PIER+ (Zeng & Bhat, 2023) . To study the extent to which scaling and strategic fine-tuning methods enable idiomatic reasoning, we also compare our proposed method with open-source LLMs, including Llama2-13b (Touvron et al., 2023) and Mistral-7b (Jiang et al., 2023) with 10 in-context examples (the number 10 was empirically found to yield reasonable results beyond which the performance did not improve significantly).Prior works (Sag et al., 2002; Tayyar Madabushi et al., 2021; Stowe et al., 2022) have demonstrated that the presence of IEs negatively impacts NLI performance. We use this as a case study to demonstrate how our proposed IE-related knowledge injection increases robustness in the presence of IEs. As shown in Table 2 , our method efficiently injects idiomatic knowledge into the model. For the models without a PEFT module (Full), our proposed model outperforms the original model by 6.18% in accuracy. Notably, there is a gain of 24.48% accuracy in the non-entailment (NE) class. We point out that a good nonentailment performance is much harder to attain compared to entailment since the model resorts to predicting all instances as entailment as discussed in (Stowe et al., 2022) . We ascribe this performance gain to the model's understanding of the IE semantics rather than relying on spurious features towards the entailment predictions, which is further verified by our ablation study in Section 8.Here we present the results of two related tasks: I2L and L2I.As presented in Tables 2, our proposed approach substantially benefits these two tasks by incorporating idiomatic knowledge. Similarly, our model achieves the highest performance. It surpasses the model trained solely on the I2L dataset by a significant margin of 3.65 points in BLEU score (82.46 vs. 86.11 ) and outperforms the model trained solely on the L2I dataset by a margin of 2.89 points in BLEU score (82.89 vs. 85.78). For both tasks, this improvement holds consistently across models with different PEFT modules, showcasing the effectiveness of our method. Considering that BLEU and ROUGE scores only focus on the n-gram overlapping, we also utilize Bertscore to evaluate our proposed method. It is shown that our model also achieves the best performance on Bertscore. The details are provided in Table 6 in the appendix.As shown in Table 2 , our method efficiently equips the model with the ability of reasoning in the presence of idioms. For the models without a PEFT module (Full), the model trained on our multi-view augmented data (+PT) outperforms the original model by 3.24% in accuracy.We note that the tasks we study (IMPLI, L2I, I2L, and FNB) contain IEs not covered by idiomatic knowledge injection, further demonstrating our model's generalizability to IEs unseen before fine-tuning. Details are provided in the Section ?? in the appendix.  
analysis: 
 We gain more insights about our multi-view data augmentation method and Idapter via further analyses.Ablation Study. In our ablation studies, we analyze the impact of excluding different components from our proposed model. Table 2 demonstrates that omitting any part of our model results in decreased performance, highlighting the effectiveness of our approach. Specifically, when the multi-view data augmentation is excluded, performance is significantly worse, emphasizing the importance of this augmentation method. Additionally, excluding either the non-compositionality scaling layer or the task-specific scaling layer leads to performance degradation, reinforcing the effectiveness of the various inductive biases incorporated in our proposed Idapter.Injection Task Influence. To study the influence of different continued pre-training tasks, we pre-train the Idapter with the absence of different tasks. For the 0-task case, we directly fine-tune the Idapter with the backbone PTLM on the downstream tasks with no pre-training injection tasks. Then we exclude different injection tasks from the pre-training data and pre-train the Idapter with the remaining two tasks. Finally, for the 3-task case, we first pre-train the Idapter on all three augmented tasks, adding IE appropriateness compared to the 2-task case. After the pre-training with augmented tasks, we further fine-tuned the Idapter on different downstream tasks. As shown by the results in Table 3 , all instances of excluding different injection tasks degrade the performance across all four downstream tasks, while pre-training with all three tasks provides the best performance. It should be noted that excluding the definition generation task impacts the performance the most, presumably because this task directly injects the meaning of idioms into the model.Another question concerns whether a model equipped with generalized idiomatic knowledge is able to learn different downstream tasks in a data-efficient way. To this end, we fine-tune the continued pre-trained Idapter on the downstream tasks with fewer data. We use only 20% of the original data in different downstream tasks for fine-tuning and summarize the results in the Table 4 . As is shown, using only 20% of the data, our proposed model achieve the best performance. In fact, the model augmented with the Idapter achieves a competitive performance that is only slightly worse than the model fully fine-tuned on the entire data for IMPLI; this confirms the data efficiency of our method.Zero-Shot Setting For IMPLI. Our multi-view data augmentation method shows an improvement in a zero-shot, out-of-distribution setting. We observe this by fine-tuning the full model only on the MNLI dataset (Williams et al., 2018) , a generic NLI dataset with a negligible number of IE instances, and test on IMPLI; models trained with the multi-view augmentation data show an improved overall performance on the IMPLI dataset (+8.04% gain in accuracy). Our method's notable improvement comes from a 1.24% accuracy increase in NE class, 2.1% in ANT class and 7.43% in the E class. Details are provided in Table 7 in the appendix.Qualitative Analyses. Some idioms may only appear in the downstream tasks, but not in our continued pre-training data. Therefore, we explore our model's performance on idioms seen during pretraining and those unseen for different downstream tasks. As shown in Table 5 , across different downstream tasks, our model was able to process unseen idioms, though not nearly as well as those seen.Beyond processing idioms, we find that our model is able to positively impact the processing of other types of figurative expressions, including metaphor detection (+0.8% on accuracy) and euphemism detection (+1.03% on accuracy). Details are provided in appendix A.4.  
conclusion and future works: 
 In this work, we propose a continued pre-training method based on a novel multi-view data augmentation approach, a parameter-efficient training method, Idapter and a metapretraining mechanism. Unlike previous works on IEs that only focus on one task, we empirically show that our method injects generic idiomatic knowledge that is beneficial for various tasks involving IEs, such as IE-related natural language inference, idiomatic expression paraphrasing, literal expression stylizing and figurative narrative reasoning. Besides, we also show that even with only 20% of the training data for downstream IErelated tasks, the model equipped with generalized idiomatic knowledge yields comparable performance with the model trained on the whole data on IE-related natural language inference. Future work should explore how this method can be useful in low-resource scenarios, such as the medical and legal domains. 
 
 Appendix: The main limitation of our method is that it still needs external resources to obtain the definitions of IEs, which restricts the learning of some IEs whose definitions cannot be easily accessed.Also, the scope of this research focuses on equipping large PTLMs with idiomatic knowledge efficiently. However, due to the limitation of our computation resources, we cannot perform our experiments on larger PTLMs such as T0 or GPT-4. However, we believe that while optimizing hyperparameters should yield higher performance, the trends between different pipelines should remain the same (i.e., knowledge injection should still be beneficial).Additionally, although the scope of this paper is to understand the effect of idiomatic knowledge injection on IE comprehension, it would be more beneficial for the community to generalize our method to more domains that require domain knowledge, e.g., medical domain, legal domain, and science domain. However, our experiments are limited by focusing only on the idiom domain.critical domains. During the training and testing of our methods and their downstream applications, we only use publicly available datasets without collecting or generating additional data that could breach privacy rights. Hence, when used properly, our work presents no calls for ethical concern.Our experiments are performed as follows: we use the T5-large model Raffel et al. (2020) as the backbone PTLM. We add Idapter to the backbone model, which is then continued pretrained to perform the idiomatic knowledge injection using our multi-view data augmentation method. For this continued pre-training stage, we train the Idapter for 5 meta-epochs within each of which the model is trained for 50 epochs with a batch size of 32 using the Adam optimizer with a learning rate of 2e-5 and a linear decay scheduler with 60 warmup steps. For outer learning rate of meta-learning is 1e-5. After injection, we deploy the model on different downstream tasks, where we use the parameters initialized from the trained Idapter. We train for 20 epochs with a batch size of 64 and the same optimizer and scheduler setup as before. We apply prompt templates shown in Table 1 to all the tasks during training and inference to convert each example into an instructive text-to-text format. Importantly, we apply this recipe to every downstream dataset in exactly the same way without modifications.IE-related Natural Language Inference. This semantic understanding task involves determining whether a given hypothesis can be inferred from a given premise (entailment) or not (non-entailment). The overall challenge is compounded with the insertion of an IE in the Idiomatic and Metaphoric Paired Language Inference (IMPLI) dataset (Stowe et al., 2022) . For this dataset, E represents entailment by paraphrasing idioms in the premise, NE represents non-entailment created from adversarial idiom definitions, and ANT represents non-entailment created by swapping keywords in idiom definitions to their antonyms.Stylizing. Proposed by (Zhou et al., 2021c) , the tasks involve paraphrasing IEs into literal phrases and stylistically transforming the literal phrases in given sentences by replacing them with their IE counterparts, respectively. We use the dataset created by (Zhou et al., 2021c) for this task (denoted as I2L and L2I henceforth).Figurative Narrative Benchmark. Proposed by (Chakrabarty et al., 2021a) , the task is to continue a given narrative by choosing an acceptable continuation from two candidates. The dataset has narratives that use idioms and similes. Following the settings in (Zeng et al., 2023) and to account for our setting of working with idioms, we only utilize the narratives with idioms. Moreover, we transform each instance into two continuation acceptability examples: one that pairs the narrative with the correct continuation, and another that pairs the same narrative with the incorrect continuation (denoted as FNB henceforth).Here we show that injecting idiomatic ability does not take away our model's prior ability with compositional expressions. We perform experiments on sentiment classification and paraphrase identification following the settings in Zeng & Bhat (2023) . For sentiment classification, we use the SST2 (Socher et al., 2013) dataset and its default train and test splits (two classes). For paraphrase identification, we combine the MRPC (Dolan & Brockett, 2005) and PAWS (Zhang et al., 2019a) datasets and their default train/test splits with a total of 53,069 train and 9,725 test instances. The experimental results are presented in Table 8 . As shown in Table 8 , compared with the original T5-large, our model still achieves a comparable performance for these two tasks, which proves that injecting idiomatic ability does not take away our model's prior ability with compositional expressions.Here we show that injecting idiomatic ability could also help with the comprehension of other types of non-compositional expressions including metaphors and euphemism. We Table 6 : Performance of different methods for i2l and l2i downstream tasks. Here we include the Bertscore for more thorough evaluation. When evaluating using Bertscore, we only extract out the idiomatic expressions and their corresponding literal parts. For I2L, we directly calculate the bertscore based on generated literal parts and reference literal parts.For L2I, we calculate the Bertscore between the generated idioms and the reference idioms. 


Decision: The work introduces a pretraining methodology aimed at enhancing language models with better idiomatic reasoning capabilities. The method involves training an adaptor using a MAML-style algorithm, specifically REPTILE, a more efficient first-order method. The effectiveness of this method is then assessed across downstream tasks that require idiomatic processing.  All the reviewers agree on the merits of the work.


However, there is a significant issue that has only partially raised during the author-reviewer discussion stage, but has been discussed between reviewers and AC in the follow-up period. We strongly encourage you to address it. The paper fails to discuss the fact that the pretraining method and at least three of the four evaluation tasks (IMPLI, L2I, and I2L) partially originate from the same source, MAGPIE. This suggests the possibility of an unrealistic overlap between the idiomatic expressions seen in pretraining and those seen in testing. Although the appendix (Table 7) presents results separately for unseen and seen PIEs, it only evaluates the proposed system in isolation. I believe that evaluating the unseen subsets against alternatives in Table 2 is crucial and needs to be included in the paper.

Additionally, it is important to openly discuss and clarify that these datasets are created from a common resource. The proportion of unseen and seen examples in the test set also needs to be mentioned. If there is no substantial improvement on unseen constructions, the claims should ideally be corrected accordingly. Currently, the approach is positioned as improving the acquisition of generic IE knowledge, which would be debatable if the improvements are observed only on 'seen' PIEs.

[comments from PCs] Please make sure to follow the AC recommendations, especially with regard to data and evaluation.

Reviews: review: paper_summary: paper_summary: This paper investigates use of idiom aware PEFT method to improve backbone model's performance over non-compositional tasks like idiom detection, idiom generation and idiom definition generation. Its essentially a training framework that is proposed in the paper each sample in the dataset is split into different tasks (multi-view data augmentation), PEFT is used to inject idiomatic knowledge in the model and meta-learning over multiple tasks is applied in order to generate a better initialized pre-trained model ideal for being trained on downstream tasks.
strength_weakness: strength_weakness: * Its unclear how many additional parameters were added because of non-compositionality scoring (W_n) and task specific vectors (W_h). Are the parameters for multiple baselines (shown in Table 2) at par with your model?
* It would be better to see some examples where the proposed approach worked and failed to identify idiom/generation idiom. 
* Nitpicking here, but results are only computed with T5-large as a baseline which begs a question - what happens when Idapter is applied to other LMs?
* From the results in Table 2, it looks like data augmentation was the most effective, so why was the paper pitched for Idapter?
questions: questions: * Conceptually, Idapter like adapter can be applied to other tasks as well (not just idiom related tasks) like style adaptation (formal/informal/semi-formal), one just needs a good dataset. Why was this work limited to idiomatic expressions? 
* Typically with idioms, its easy to memorize them during training so generalization (over non-literal data) is quite difficult. Appendix provided necessary details like performance on seen/un-seen idioms - consider adding this table in main paper. Can you provide examples of failure and success cases in Appendix for readers?
* Do-no-harm experiments (no regression on standard tasks) are important for productionizing any work. Table 8 is quite important in that aspect, maybe find a place to reference section A.4?
paper_summary: paper_summary: This paper introduces a data augmentation method, a model adapter, and a training scheme to first enhance a pre-trained language model (PTLM) with idiomatic knowledge, and then fine-tune it for idiomatic-aware tasks. 

Below, I describe the three components of the proposed framework:

1) **Data Augmentation**: Their approach for data augmentation consists of taking an existing dataset of idiomatic expressions and create 1-1 examples for three different tasks: 1) generating dictionary definitions of idiomatic expressions, 2) identifying expressions in a sentence that are used in their idiomatic sense, while ignoring literal senses and 3) a cloze task that masks phrases and recovers appropriate idiomatic expressions from their context. This multi-task views allows to learn different aspects for the same input (and idiom).

2) **Network Adapter**: Their adapter includes a layer that aims to capture non-compositional information, and a derived scaling factor that allows to balance the original representation and the induced "non-compositional information" during training and inference. They also learn a scalar vector for each tasks to adapt to them. 

3) **Meta-pretraining**: This component was quite confusing to me. Authors claim it is supposed to consider aspects of the downstream tasks (but it is not standard fine-tuning?) and I don't understand why. In general, it wasn't clear to me what they do here. What are the meta-tasks? Are they different from the pre-training tasks AND the downstream tasks? How do they alleviate the disconnect between pre-training and downstream tasks?

Finally, the authors perform an experimental evaluation on a diverse set of downstream tasks (tasks that require deeper understanding of idiomatic expressions) and show that their model outperforms full fine-tuning, standard parameter efficient fine-tuning and in-context learning for most tasks.
strength_weakness: strength_weakness: 1) My only real issue with this paper is that the learning scheme is a bit unclear. I think the paper could benefit from a re-write of the meta-pretraining section (see comments and questions in summary above)

2) Maybe it is me nitpicking, but I am not sure some of the names chosen for the parameters are completely justified. How can we know for sure that the added layer is capturing non-compositional information? All we can see is that it contributes to the improved performance (as seen in the ablation study) but this is not really enough evidence of this parameters capturing this information. Calling it non-compositional layer reads like a soft claim that isn't actually validated.
questions: questions: 1) I suggest that you bold the highest numbers in Tab 3 as you do in the other tables, it is a bit confusing to just see the last line bolded even in cases where the performance decreases (E for IMPLI)
paper_summary: paper_summary: The paper presents a new approach to improving the handling of idiomatic expressions by pre-trained language models. It highlights the difficulty that existing large language models (LLMs) face with figurative language, which often deviates from standard compositional rules, making idiomatic expressions particularly challenging to interpret. The authors introduce a method that integrates a multi-view data augmentation strategy to generate diverse training scenarios from a single dataset, focusing on different aspects of idiomatic usage. This is combined with a parameter-efficient tuning method and meta-pretraining techniques that tailor the model's learning process to enhance its idiomatic reasoning capabilities. Empirical tests demonstrate that this approach not only outperforms several baselines in tasks involving idioms but also enables the model to adapt more efficiently to various downstream tasks requiring the understanding of idiomatic language.
strength_weakness: strength_weakness: 1. The research relies predominantly on the MAGPIE dataset, which might not cover the full spectrum of idiomatic expressions encountered in natural language, potentially limiting the model’s effectiveness with unfamiliar idioms. 
2. The experimental setups are not entirely clear. For example, what is the rank of LoRA? I didn't see the trainable parameter size of each compared model in Table 2. In this context, it is not clear whether the improvements are comparable within similar trainable parameter size. The test data size (#samples) of the downstream tasks is also not provided (nor in the appendix). Also, it would be better to also analyze the training efficiency of other PEFT methods.
3. The proposed method is not tested under broader tasks, such as text generation, etc. So it is doubtful if the proposed method can be applied to some real application scenario of LLMs (not just doing some classifications or simple text extractions). As the proposed PEFT method and meta-pretraining mechanism seem to be applicable to other general NLP tasks, adding more types of tasks would have strengthened the generalizability of the proposed method.
4. The proposed method is not performed on bigger LLMs, such as Llama, and Mistral, where the authors only perform on T5-large. It is questionable whether the proposed method can still be effective on a stronger baseline.
questions: questions: Please address the weaknesses illustrated in the previous section.



Other comments:

1. PTLM seems not to be a common abbreviation for pre-trained language model. => PLM
2. Section 1: LlaMa2 -> Llama-2 or Llama 2
3. Section 2: missing year info for (Hu et al.)
4. Section 6 baselines: Lora -> LoRA
paper_summary: paper_summary: This is a good paper introducing a new fine-tuning method to inject idiomatic knowledge into pre-trained language models. The author(s) propose(s) a multi-view data augmentation strategy that automatically converts a single-task dataset into labeled data for three tasks, and variations of parameter efficient and meta-learning to improve the fine-tuning process. It is worth to mention that after reading the paper several times, I could not figure out in which language the experiments were done, or whether the author(s) believe that the approach is language-independent, in the sense that with labeled data (like MAGPIE) it would work in a large number of languages.

The paper is well motivated, has a clear structure and it is easy to read. The related work is not bad, and the experiments seem solid. They include competitive baselines, an ablation study, and additional experiments shown in the appendix. I lack information about the availability of the code: without code and data, the experiments are not replicable, and without replicability science does not progress.

I miss some general evaluations of the proposed model vs. the original T5 on non-idiomatic tasks. This would inform the readers about the performance of the model in both scenarios (compositional and non-compositional). In this regard, I assume that the objective is to have a good model that adequately represents "multiword expressions", both compositional or not, and not focus only on idioms.

The analysis in Table 7 (performance in seen vs. unseen idioms) is interesting. It would be nice to include in this table the results of the "Full" model, in order to compare the effect of the proposed strategy for seen/unseen cases.

As an aside, I wonder to what extent we can rely on a dataset like MAGPIE (which is a great resource!) for data augmentation. For instance, in the "Idiom identification" example of Table 1, "catch a cold" is the only expression in this sentence labeled as an idiom. It can certainly be considered an idiom, but I would say that most scholars would classify it more as a "collocation/light verb construction", and only partially idiomatic ("cold" conveys its standard meaning in "catch a cold", right?). Moreover, an expression like "looks like" could also be considered idiomatic, and "temperature [...] drop" metaphorical. With the data augmentation technique we lose this information, and implicitly tell the model that these are compositional (non-idiomatic, non-metaphorical) expressions.

Finally, I would remove the last sentence of the conclusion ("our method can be applied to [...]"): if you have already applied your method to other domains, please say so and comment on some preliminary results. Otherwise, just saying that it could be applied to other domains is not informative, unless you have some hypotheses about the adaptation process or about the expected performance.
strength_weakness: strength_weakness: I do not see solid reasons to reject the job. Some minor issues and doubts that can be seen in my review, but I think the paper should be accepted.

Comments: title: Thank you.
Acknowledgement
comment: R1. Thank you for your clarification. 

R2. I understand the inspiration for the name, but I will challenge that it is not necessarily clear whether this information ends up being captured in this layer.
Thanks for the response. I'll stick with my rating.
Thanks for the rebuttal. I keep my original rating.