A Couch Potato is not a Potato on a Couch: Prompting Strategies, Image Generation, and Compositionality Prediction for Noun Compounds (2025)

Abstract: We explore the role of the visual modality and of vision transformers in predicting the compositionality of English noun compounds. Crucially, we contribute a framework to address the challenge of obtaining adequate images that represent non-compositional compounds (such as “couch potato”), making it relevant for any image-based approach targeting figurative language. Our method uses prompting strategies and diffusion models to generate images. Comparing and combining our approach with a state-of-the-art text-based approach reveals complementary contributions regarding features as well as degrees of abstractness in compounds.

## introduction compositionality represents a core concept in linguistics @xcite : the meaning of complex expressions, such as compounds, phrases and sentences, can be derived from the meanings of their parts. the degree of compositionality however varies; e.g., while the compound climate change has a high degree of compositionality, couch potato is less so regarding its constituent potato, because it does not refer to a potato lying on a couch. for natural language understanding tasks such as summarization, machine translation and retrieval systems, the accurate prediction of compositionally is crucial to ensure precise and reliable results. the focus of this paper is on predicting degrees of compositionality for english noun compounds. in contrast to state-of-the-art models, which primarily leverage text-based representations to assess the relatedness between compound and constituent meanings (see section 2), we explore the contribution of the visual modality, which previously has proven successful across semantic tasks @xcite @xcite @xcite . applying vision models to any task involving non-compositionality however comes with the major challenge of finding appropriate images, because standard image retrieval methods return false positives for noncompositional expressions, e.g., a couch potato is actually depicted as a potato (instead of a lazy person) sitting on a couch, cf. bing (left) in figure 1 . the current study offers a novel way of obtaining "correct" images, which we judge highly valuable for any vision work involving figurative language: we carefully design and compare prompts as input for an image generation model, in order to obtain adequate images for both compositional and non-compositional compounds. the actual compositionality prediction then follows standard routes, i.e., estimating the degree of compositionality via similarity of compound and constituent feature vectors. evaluation is carried out by measuring the rank correlation between similarity estimates and human ratings. in addition to our main contribution of (i) prompting strategies with increasing contextual description levels to obtain images of non-compositional expressions, we conduct analyses to identify aspects relevant for vision models, including (ii) the role of abstractness, given that abstract concepts are generally more difficult to depict than concrete concepts @xcite , and (iii) the role of meaning prototypicality. finally, (iv) we compare our visual approach against a state-of-the-art text approach, a multimodal approach, and chatgpt predictions. ## related work traditionally, most computational approaches to automatically predict the compositionality of noun compounds have been realized using text-based vector space models by comparing compound representations with those of individual constituents or a combined representation @xcite @xcite @xcite . few studies addressed compound meaning using multimodal information; bruni et al. (2014 ) identify figurative uses of color terms in adjective-noun phrases, pezzelle et al. (2016) and günther et al. (2020) predict compound representations using constituent-based text and vision features. roller and schulte im walde (2013) and köper and schulte im walde (2017) represent two rare previous cases of multimodal studies predicting compositionality of german noun compounds, by relying on a multimodal lda model and textual plus visual vector spaces, respectively. ## our methodology given a compound (e.g., couch potato), our task is to assess how related the compound meaning is in relation to the meanings of the constituents, i.e., the modifier (couch) and the head (potato), by relying on reliable images. ## conclusion this study explored the contribution of the visual modality to the prediction of compositionality for english noun-noun compounds, focusing on the challenge of obtaining adequate images, especially for non-compositional compounds, by providing prompting strategies for generative models with increasing contextual description levels. we further analyzed especially challenging sub-cases, such as abstract targets and meaning prototypicality, as well as complementary distributions of visual and textual information.