METAPHORSHARE : A Dynamic Collaborative Repository of Open Metaphor Datasets (2025)

Abstract: The metaphor studies community has developed numerous valuable labelled corpora in various languages over the years. Many of these resources are not only unknown to the NLP community, but are also often not easily shared among the researchers. Both in human sciences and in NLP, researchers could benefit from a centralised database of labelled resources, easily accessible and unified under an identical format. To facilitate this, we present MetaphorShare, a website to integrate metaphor datasets making them open and accessible. With this effort, our aim is to encourage researchers to share and upload more datasets in any language in order to facilitate metaphor studies and the development of future metaphor processing NLP systems. The website has four main functionalities: upload, download, search and label metaphor datasets. It is accessible at www.metaphorshare.com.

## introduction the topic of figurative language processing has been addressed since early years of artificial intelligence research @xcite @xcite , inspired by philosophical @xcite , linguistic @xcite and then cognitive science theories @xcite and experimental studies @xcite . in spite of the influence of the different disciplines on one another, the resources created to train and evaluate nlp models have often been distinct from the resources created for metaphor studies in other fields, with the notable exception of master metaphor list @xcite and the vu amsterdam metaphor corpus @xcite . in recent years, metaphor processing has attracted more and more attention with the progress made possible by transformer-based language models, for example @xcite or @xcite , and in particular the large lms such as @xcite used by @xcite for metaphor identification. several dedicated workshops with shared tasks @xcite @xcite have been organized, leading to the creation of more resources. the proper handling of figurative language by models is of crucial importance for improving performance in downstream tasks @xcite . from the perspective of humanities, tools for searching corpora such as sketch engine @xcite , annotation tools @xcite and data analysis softwares have become part of the standard corpus-based research methodology. effective adaptive support for metaphor identification and interpretation is the next step of this on-going inter-disciplinary collaboration. there were two limitations until recently: metaphor identification systems were not accurate enough to be used on free text, and different researchers are looking for different metaphors. on one hand, numerous datasets labelled with metaphoric usage of words are created by researchers in many languages every year, and remain out of the scope of nlp research, tool development, and evaluations of the systems. this happens because most datasets are published in non nlp venues, they might not be publicly available, or might be tagged in an under-specified format that does not allow direct automatic processing (e.g. tagging precisely a metaphoric expression within a text). on the other hand, metaphor studies researchers often work on domain specific corpora, languages other than english -for which nlp has developed the most resources -and use different definitions or retain different types of metaphors for different projects, with no convenient way to systematically compare their labels with other researchers, or to rely on an existing metaphor automatic labelling tool that answers their project specific need. we propose to help speeding up collaborations between ai/nlp communities and linguistics/metaphor studies by facilitating the unification of dataset formats and the access to existing resources by everyone, while preserving the information encoded in the original datasets. metaphor-share is a website designed at sharing new and publicly available existing resources (see figure 1 ). it is an online dynamic repository where anyone can upload or download open metaphor datasets. our online labeling tool also offers the possibility to directly create a dataset, and optionally share it. our hope is that the website will create a new synergy between different components of metaphor studies, limit the loss of datasets, catalog them, and facilitate their automatic processing through a unified format. our target audience comprises anyone conducting corpus-based research on metaphors, including nlp researchers for the development of metaphor processing and identification systems 1 . recent advances in metaphor processing are presented further in section 2. the diversity of existing resources and the unified dataset input format is presented section 3. the website is structured around four main functionalities that are described in section 4: upload, download, search and label metaphor datasets. section 5 shows the potential of our framework for facilitating nlp experiments, by evaluating a roberta @xcite ) model on a cross-dataset classification task. metaphor related websites. @xcite develop a system of metaphor interpretation and generation, metaphor magnet, that relies on the harvest of stereotypes from google n-grams. the stereotypical attributes associated with concepts are leveraged for the suggestion of relevant metaphors and their interpretation (e.g. given google is -microsoft, the system outputs giant with properties like lumbering and sprawling). more recently, @xcite release an end-to-end domain independent metaphor identification and interpretation website for free text: metapro tags metaphoric tokens or multi-word expressions in sentences, generates paraphrases, and outputs a more abstract concept mapping derived from the analysis of the sentence. the authors list limitations due to the lack of training data for inference and extended metaphors. the main difference with these two initiatives and ours is that our aim is to integrate already labeled datasets into a unified repository, encouraging the structuring of resources and expression of needs from the metaphor study community, before building automatic annotation tools. similarly to ours, metapro also uses several metaphor datasets for training the models @xcite , but in this case they do not mention more than six datasets, all of them often used by the nlp community. annotation tools. many open source (e.g. label studio, dataturks, doccano @xcite and potato @xcite ) and proprietary (e.g. labelbox, prodigy or amazon mechanical turk) annotation tools exist. our purpose is not to compete with these services but rather to offer a simple targeted experience for metaphor annotation, in which the output is directly mapped to the unified repository format. this way, non-expert users can avoid the need for learning a general tool, as well as having to learn how to modify given labeled outputs, which would not be trivial for most. metaphor processing in nlp. one important usage of our website is to facilitate the creation of personalised metaphor processing models. metaphor processing in nlp comprises many methods developed for metaphor identification @xcite @xcite , but also generation @xcite @xcite , textual @xcite and multimodal @xcite interpretation, metaphor understanding through entailment @xcite @xcite , among other tasks. @xcite provide a comprehensive recent survey on the topic. ## data sharing metaphorshare aims at facilitating computational research on multiple datasets. a minimally constrained format is required to store the datasets into a database, conveniently compare the resources, search efficiently into the different fields across datasets, and run experiments on several sets name reference n %m words in syntactically constrained sentences (psycholinguistics) jank jankowiak (2020) 240 50 @xcite 280 50 @xcite 512 50 moh mohammad et al. (2016) 1632 25 newsmet joseph et al. (2023) 1205 49 tsv_a tsvetkov et al. (2014) 1945 50 gut gutiérrez et al. (2016) 8591 54 mwe in natural long context (nlp) pvc tu and roth (2012) 1348 65 mad tayyar madabushi et al. (2022b) 4558 48 magpie haagsma et al. (2020) 48395 75 words sampled from vuac (mipvu) vuac_bo boisson et al. (2023) 39223 52 tong tong et al. (2024) 1428 46 at once. metaphor datasets can be added by their authors or by anyone else if their license allows it. in this section, we review the existing dataset formats and our unification choices. the website database currently contains 25 datasets. we select twelve datasets with open licenses and diverse encoded information to illustrate our unification process and as representative examples for our repository. their characteristics are summarized in ## evaluation as a sanity check, we first verified that metaphor-share uploading, database insertion and search functionalities are working properly with the datasets presented in then, as a case study we perform a cross-dataset metaphor identification analysis to illustrate a possible usage of metaphorshare for nlp research. given an input expression, the task consists of deciding whether the expression is a metaphor or not. different projects label metaphors for different purposes. metaphorshare makes it easy to fine-tune models on specific datasets to further support project-specific automatic or semi-automatic labeling. experimental setting. in order to maintain a training set of the same size for all datasets, we randomly sampled 800 examples from each set to create the training data. because our datasets from psycholinguisics are small and similar in their creation method, we grouped them into one single set called j&c in this experiment. roberta base models @xcite are then finetuned independently on the 10 datasets and tested on each of them. similarly to the experiments in @xcite , for hyperparameter optimisation, we rely on the bayesian optimization with the hyperband (bohb) algorithm @xcite available in raytune @xcite , with an identical hyperparameter search space and 25 trials. results. results are shown in figure 3 . as expected, models trained and evaluated on the same dataset often achieved the best results. a few datasets generalise better than others. it is surprisingly the case for j&c and gut, both containing only short example with constrained syntactic structures. on the other hand, datasets such as tong and news do not generalize as well, probably due to different labelled pos, to the token or mwe span of the labels, or due to redundant contexts corresponding to datasets created for other tasks such as paraphrasing in the case of tong. ## conclusion and future work in this paper, we have presented a website to label, unify and share metaphor datasets. it enables an easy integration of new resources of different original formats by the community. there are currently 25 english datasets integrated in the platform. in the future, we are planning to integrate and reach communities working on other languages as one of the main aims. as far as the platform is concerned, we will focus on integrating a functionality for automatically labelling raw text. the progress recently made by transformer-based language models for metaphor identification opens the path to the creation of helpful personalized tools for automatic or semi-automatic labelling of metaphors based on an initial sample of annotations.