Evaluating Textual and Visual Semantic Neighborhoods of Abstract and Concrete Concepts (2025)

Abstract: This paper presents a systematic evaluation of nearest neighbors across semantic representation spaces in both textual and visual modalities. We focus on nominal concepts with varying concreteness levels, and apply a neighborhood overlap measure to compare these target concepts differing in their linguistic and perceptual nature. We find that alignment is primarily determined by modality, and additionally by level of concreteness: Models from the same modality show stronger alignment than cross-modal models, and spaces of concrete concepts show stronger alignment than those of abstract ones. Overall, larger neighborhood size strengthens the alignment between spaces.

## introduction humans possess an intuitive understanding of concepts and their relative positions within a shared semantic space. for instance, we perceive cat as being more similar to dog than to table. this sense of conceptual proximity is grounded in real-world knowledge and similarity across multiple attribute dimensions-such as form (e.g., four-legged animals), category membership (e.g., pets, furniture), and function @xcite . in natural language processing, prototypical attributes and similarity between concepts are generally captured by vector representations and vector distance measures. at the textual level, these representations are learned either through count-based methods such as co-occurrence matrices @xcite , or predictive models such as shallow neural networks trained on target or context prediction objectives @xcite . in these vector spaces, relative position and structure encode semantic relatedness. similarly, vision models such as convolutional neural networks @xcite and vision transformers @xcite ) map images into vector spaces where proximity reflects ## data and methods in this section, we describe the target concepts, the variants of textual and visual representations, and how we identify nearest neighbors. target concepts this study uses a curated set of 1,500 nouns drawn from the brysbaert concreteness ratings @xcite . to guarantee clear distinctions between sets of concepts, this set consists of 500 nouns each from three distinct levels of concreteness: extremely abstract, intermediate mid-scale, and extremely concrete noun concepts (see ## experiments and results given that we are interested in overlapping vs. complementary nearest neighbors of abstract and concrete concepts in various modality spaces, we report as results and main insights variants of pairwise comparisons relying on nas. effect of models and modalities figure 1 shows nas scores for all model pairings across the three concept sets (abstract, mid-scale and concrete, from left to right) at a fixed neighborhood size of 25. the top and left-most four models in the matrices refer to the visual representations, the bottom and right-most seven (four distributional word and three sentence representations) refer to the textual ones. modality is clearly the overall dominant factor in alignment scores, i.e., across concreteness levels, model pairs from the same modality (text-text and image-image, cf. top left and bottom right parts of matrices) show substantially stronger overlap than cross-modal pairs: across abstract/mid-scale/concrete concept sets, alignment scores of image-image spaces reach 0.50/0.56/0.58 (vit/dinov2), and alignment scores of text-text spaces reach 0.69/0.73/0.77 (word2vec/fasttext), respectively, while across-modality alignment scores only reach 0.14/0.16/0.26. the architectural differences of models within a modality also influence the alignment (with strongest within-modality alignments for wordbased distributional models, and weakest ones for sentence-based representations), however to a lesser degree. overall, the presented alignment patterns thus indicate that while semantic spaces are structured similarly within the same modality, visual and textual modalities shape them in fundamentally different ways. the modality gap is strongest for abstract concepts, which are inherently more challenging to ground in visual features. for example, the 3×3 top left parts of the matrices in figure 1 ( image-image alignment) show increases from alignment range [0.25, 0.50] for abstract concepts to [0.36, 0.56] for intermediate and [0.44, 0.58] for concrete concepts; the 4 × 4 middle part of the matrices (distributional text-text alignment, which also presents the overall strongest alignments) shows increases from alignment range [0.50, 0.69] for abstract concepts to [0.54, 0.73] for intermediate and [0.58, 0.77] for concrete concepts. we also find -but to a more subtle degree -that both within and across modalities the alignment is consistently higher for concrete concepts in comparison to mid-scale concepts, where it is again higher than for abstract concepts. figure 2 illustrates this pattern across concreteness levels, by showing the differences (∆ scores) between alignment scores of mid-scale vs. abstract concept sets (left), concrete vs. mid-scale sets (middle) and con-crete vs. abstract sets (right). we can see that the ∆ scores strongly increase from left to right, with differences of up to 0.21 for image-image alignment and up to 0.18 for text-text alignment. this confirms prior research that concrete concepts tend to have less diverse neighbors than abstract concepts @xcite @xcite @xcite @xcite . ## conclusion we presented a systematic evaluation of nearest neighbors of abstract and concrete concepts by applying a simple, interpretable, modality-and modelagnostic metric to comparing a variety of textual and visual semantic embedding spaces. our results confirm that modality is the primary factor shaping semantic structure: alignments of neighborhoods within the same modality are stronger than alignments across modalities. specifically focusing on concepts across concreteness levels, we found that concrete concepts show higher alignment of semantic space neighbors than abstract ones, which confirms the difference in perceptual strength in the visual domain, where grounding them is especially challenging. a mean aggregation of images strengthened the alignment with diminishing returns beyond 20-25 images per concept; also, larger neighborhood sizes evoked stronger alignments. our findings provide a foundation for further analysis of cross-model and cross-modal differences in meaning representation.