DEIM Forum 2023 4a-1-3

Visual Metaphor Generation based on Similarities
in Appearance and Concept
Dan WANG† , Ryota MIBAYASHI† , and Hiroaki OHSHIMA†† Graduate School of Information Science, University of Hyogo
8–2–1 Gakuen-nishimachi, Nishi-ku, Kobe, Hyogo 651–2197, Japan
E-mail: †ad21h036@gsis.u-hyogo.ac.jp, ††threeforest8@gmail.com, †††ohshima@ai.u-hyogo.ac.jp
Abstract In this paper we propose a framework for automatically generating visual metaphors based on similarities on visual and conceptual features. We put the idea into practice for generation of visual metaphors for fashion
goods. For example, for a red jacket with material with good heat retention, the visual metaphor generated would
be ﬂame. Flame and product have same visual features of color and are consistent in the concept of “giving a warm
feeling”. In order to explore the patterns of establishment of visual metaphors based on human views, we build
visual metaphor dataset for fashion goods manually, which contains pairs of fashion goods images and metaphor
images. Based on this dataset, we propose visual metaphor generation methods based on color, shape, and texturalsimilarity. Finally, we evaluate visual metaphors in terms of both appearance and concept, then discuss the uses ofgenerating catchphrases for fashion goods based on visual metaphors.
Key words feature extraction, image recognition, clustering, search model

1.

Introduction

Visual metaphor, which is mainly discussed in this study,
is a metaphor that is established based on visual similarity.
The girl’s face is as red as an apple is an example of visual
metaphor. This visual metaphor is valid because the girl’s
reddened face in shyness looks like a round, red apple fromthe appearance. In addition to the similarity in appearance,
the girl’s face and apple also have conceptual similarity. We
often say that a girl’s face is cute, and the adjective “cute”
is also used to describe an apple. This is the conceptual
similarity between a girl’s face and an apple.
This paper attempts to challenge the generation of visual
metaphors based on visual similarity and conceptual similarity. We try to propose a methodology on visual metaphor
generation that can be applied to anything, so that relevant

Figure 1 Example of input and output for this study

researchers can refer to our framework and implement visual
metaphor generation in various ﬁelds. In this paper, we takeruby. When imagining a ruby, each person may have a dif-fashion goods as the research object. We aim to automat-

ferent image in mind, but the color and texture of the rubyically construct visual metaphors based on product images

are common to all. Therefore, when this product is com-and descriptions. Figure 1 shows examples of input and out-

pared to a ruby, it resonates with anyone. The purpose ofput of this study. A bright and shiny red handbag withthis study is to automatically generate such visual metaphorsa product description text that emphasizes a premium feel.

based on the images and descriptions of the goods that can

The visual metaphor generated for this item is the word ruby.

be understood by the public.

Ruby is a very appropriate visual metaphor for this product,

Using visual metaphors to express the characteristics of

because the shape, color, texture and other characteristics

fashion goods can achieve a very good eﬀect to consumersof the product are consistent with the general impression of

in product promotion. However, it is not an easy task fordesigners to ﬁnd a appropriate visual metaphor for a fashion good. It takes a lot of imagination and knowledge to
conjure up a visual metaphor that resonates with the public
and reﬂects the visual features of a fashion good well. Therefore, we propose the method of visual metaphor generation
to solve this diﬃcult task and to support the understandingof visual metaphors and their application in various ﬁelds.

2.

Related WorkWe analyzed the related research for three aspects: (a)studies of visual metaphors in advertising, (b) studies about

Figure 2 Fashion goods visual metaphor dataset

metaphor datasets. (c) studies of visual metaphor construction in various ﬁelds based on deep learning.
2. 1.

Visual Metaphors in Advertising

Visual metaphors are often used in advertising because of the visual patterns and conceptual meanings they
carry to enhance the persuasiveness of the message [10] [1].
Forceville’s research on visual metaphors cites many formsof visual metaphors used in advertising [7]. Among them,visual metaphors are presented through various forms of media such as cartoons, comics, and posters. The promotionaleﬀect of using food as a visual metaphor for non-food brands
was also highly evaluated [13]. Due to cultural and linguistic
diﬀerences, using textual advertising in an international environment brings limited promotional eﬀect, while pictorial
advertising can be simply understood by people in diﬀer-

Figure 3 Data collection process for visual metaphor dataset

ent countries [3]. The positive eﬀect of visual metaphors on
advertising has also been mentioned in many recent stud-

ad in order to properly understand the metaphorical mes-ies [4] [20].

sage in the ad [19]. Ideation support tool is also proposed to

2. 2.

inspire visual metaphor ideas and for streamlining develop-

Metaphor Dataset

The proposed metaphor datasets in previous studies wereclassiﬁed into three categories: textual, image, and textimage categories. A dataset that can be used for discrim-

ment through multi-dimensional example exploration [11].

3.

Visual Metaphor Dataset

inating whether a syntactic construction is meant literally or

Fashion goods visual metaphor dataset deﬁned in thismetaphorically using lexical semantic features of the words

study contains paired data of fashion goods and metaphors.

that participate in the construction is proposed [17]. the VU

As shown in Figure 2, dataset has three columns, which are

Amsterdam Metaphor Corpus (VUAMC) [15], TroFi Ex-

fashion image, metaphor word, metaphor image.

ample Base [2], and MOH-X [12]. MultiMET is a dataset
including both text and images to facilitate understanding
metaphorical information [18].
2. 3.

Visual Metaphor Construction

3. 1.

Data Collection

Data of visual metaphor dataset is collected manually.
Worker is shown a large number of images of fashion goods
and then provides metaphor words and metaphor images.

Visual metaphor construction has been studied in a vari-

The ﬂow chart of data collection is shown in Figure 3. Thereety of diﬀerent ﬁelds. Model that can distinguish metaphor-were 15 workers that participated in data collection. Each

ical street art images from other rhetorical ﬁgures is sug-

worker was assigned a folder containing 800 fashion im-gested [14].

There is also method that can understandages and an answer sheet. Fashion images assigned to eachthe meaningful patterns conveyed in artworks for artwork

worker were diﬀerent. Workers selected the images that theyclustering [6]. There are also studies about uses of visual

could associate with metaphor words and recorded their ids,

metaphor based on deep learning in advertising. There ismetaphor words and metaphor images on the answer sheet.

a framework proposed for ads understanding. The frame-

We suggest the following instructions for associative

work considers both the visual and textual elements of theTable 1 Visual metaphors collected based on color similarity
fashion image

metaphor image

metaphor word

Table 2 Visual metaphors collected based on shape similarity
fashion goods

visual metaphor metaphor words

grape

browniemarshmallow

rice cracker

sky

egg

cucumber salad

metaphor words: (a) Metaphor word can be anything other

UFO

Table 3 Visual metaphors collected based on texture similarity
fashion goods

visual metaphor metaphor words

than something related to fashion (b) Metaphor word shouldhave a general and speciﬁc impression. Workers were asked

cottonto associate metaphor words based on the above two instructions. Metaphor image is the image of metaphor that is con-magma

sistent with the impression that comes to mind when workers look at a fashion goods. We asked workers to use image
search engines to ﬁnd metaphor images and to record image

gold barurls. We gave the following two instructions for workers tocollect metaphor images: (a) Metaphor images can be in any
macaroon

form, including photographs, illustrations, paintings, and soon. (b) Metaphor images should be without any impurity
such as logos, watermarks, and so on.
The data source for fashion images is from Rakuten Ichiba（注 1）

corresponding to the cucumber salad have multiple colors,. We select items from the categories of women’sand even if the colors are not perfectly consistent, the con-fashion, men’s fashion, shoes, bags, and hats as the images

sistency of the pattern can lead people to assume that theof the products that we show to the workers during datavisual metaphor is valid. This similarity is so strongly per-collection.

suasive that there are many examples in the dataset where

3. 2.

other visual features can be ignored as long as color consis-

DatasetData Study

We collected 2,362 pairs of metaphor images and fashion

tency holds. We can see the visual metaphor pairs shown inimages, and 1,049 words of visual metaphors. We divided

the table, which do not hold true in terms of similarity of

it into training data and validation data in the ratio of 8:2.

shape or texture.

Then, we study the training data to explore the conditions

Beyond color, we found that people focused on the similar-

under which the human viewpoint-based visual metaphor

ity in shape between the goods and the metaphor (Table 2).

holds.

Shape-based visual metaphors sometimes hold without rely-

A typical visual similarity pattern that we ﬁrst observed

ing on color similarity (UFO and egg shown in the table),from the training set was color. As shown in Table 1, goods

while sometimes they require both color similarity to holdand metaphors have similarity in color or pattern. Grapes(brownie and rice cracker shown in the table).

and marshmallow in the table are the simplest color-based

Another similar pattern is texture-based (Table 3). Fash-

visual metaphors. Both the goods and the metaphors are

ion goods have a variety of textures due to diﬀerent mate-consistent in a single color. Color similarity is not only re-rials, ranging from warm-looking plush, to shiny, premium

ﬂected in the same color, but also in the arrangement (pat-materials, to silky leather. Texture-based visual metaphorstern) of colors, as in the example sky and cucumber salad

are considered to best characterize fashion goods and bringin the table. In this case, the goods corresponding to theexcitement to consumers. Texture-based visual metaphors

sky are consistent in both blue and white colors. The goods

often rely on similarities in color. Inconsistencies in color
can make textural similarities diﬃcult to understand.

（注 1）：https://rit.rakuten.com/data_release/

We collected 1,049 metaphor words as visual metaphor

Table 4 Metaphor words
category

metaphor words
sea, sky, ﬁre, sun, universe, starry sky, forest,

naturaldesert, snow, rainbow, red leaf, water
ruby, emerald, sapphire, pearl, gold, cinnabar,

jewel

silver, lapis lazuli, opal, obsidian, alexandrite
chocolate, caramel, marshmallow, ice cream,

sweetswhipped cream, chiﬀon cake, chocolate cakehorse, dolphin, rabbits, penguins, butterﬂies,

animalswans, wolf, shark, tiger, swallowtail butterﬂy
milk coﬀee, milk chocolate, soda ﬂoat,

drink

cream soda, chocolate shake, orange milk
cherry blossom, rose, carnation, poppies, tulips,

ﬂowersmarigolds, hydrangea, lilies, rape blossoms
lemon, strawberry, peach, banana, apple, orange,

fruitsdragon fruit, raspberries, plums, pears, muscat

Figure 4 Training data for color, shape, texture classiﬁers

candidates. Table 4 shows some examples of metaphor candidates.

4.

Visual Metaphor Generation

In this section we illustrate the method of extracting visualand conceptual features from fashion goods and metaphor
candidates, then describe how to integrate the two similar-

Figure 5 Siamese network

ities to ﬁnally output a visual metaphor words ranking for
fashion goods.
4. 1.

Visual Feature Extraction

In Section 3. 2, we mentioned that similarities on color,shape, texture dominate the similar patterns in the visual
metaphor dataset, so we build three classiﬁers based on Inception V4 (pre-trained by ImageNet) that be able to understand basic visual feature of various objects [16] [8]. We
deﬁne 16 labels for color, 9 labels for shape and 7 labels for
texture, which are shown in Figure 4. We used each label asa query and collected more than 300 images for each label
by Bing image search API. These collected images are used
as for training data as shown in Figure 4. We remove thefully connected layer of classiﬁers to obtain feature networks
that be able to extract color, shape, texture feature vectorof images.
Further, we perform a second ﬁne-tuning based on thisfeature model. The purpose of this ﬁne-tuning is to enable
the model to associate metaphors according to human thinking. Training data that used for second ﬁne-tuning is visual
metaphor dataset that mentioned in Section 3. We ﬁrst split
the 1,889 pieces of training data into a training set and avalidation set by 8:2, and then construct separate negative
sets based on them. We ﬁrst draw out all the fashion images in the dataset and re-match each image with a diﬀerent
metaphor image than the original one. The metaphor image
is randomly selected from all metaphor images in the visual

metaphor dataset. Using this method we obtain a negative
set of the same size as positive set. The network we choose
is Siamese network based on Inception V4. Siamese network
is constructed as shown in Figure 5, whose input is a pair ofimages that are fed into the same base network to obtain thefeature vectors, respectively. We calculate the cosine similarity of these two vectors and pass Mean Squared Error forback propagation.
Following two ﬁne-tuning, we obtained three visual feature
extraction models. The visual feature extraction of fashion goods is to input the image of fashion good into thevisual feature extraction model and output visual feature
vector of fashion goods. As for visual feature extraction of
metaphor word, we use a method that referring to DeepCluster [5]. First, we collected 40 images for each metaphor
word using Bing Image Search. For each metaphor word,
We use three visual feature models to get the feature vectors of the 40 images. We perform K-means clustering on
shuﬄed 40 feature vectors. The number of clusters is ﬁxed
to 6. The images closest to the center points is selected as
representative impressions of visual metaphors. The visualfeatures of the metaphorical words are obtained by averaging 6 feature vectors of representative images. The numberof clusters and the mean-based integration method of visual feature we chose were determined based on validation of
the metaphor-generated performance. The process of valida-

Figure 7 Overview of Metaphorical Appropriateness C

in this section. which are based on single visual similarity,
double visual similarity, and three visual similarities.
Visual metaphor generation based on single visual featuremeans that one of the visual features of color, shape or texture is emphasized in visual feature extraction and the visual similarity of the fashion goods and all visual metaphorscandidates is calculated using the extracted visual feature
Figure 6 Process of conceptual similarity calculation

vector.
Visual feature extraction based on two visual features

tion is to evaluate MMR (Mean Reciprocal Rank) of visual
metaphor generation on test set (mentioned in Section 3. 2)for diﬀerent number of clusters (3, 6, 9) and two feature integration methods (mean and maximum).
4. 2.

Conceptual Feature Extraction

Calculation of the conceptual similarity between fashion
goods and visual metaphor is shown in Figure 6. First, wecollected adjectives of fashion goods and metaphor words
from web articles. For fashion goods, we used their category names (e.g., sweater, sandals) as search terms and collected articles using Bing web search API. For metaphors,
we use metaphor words as search terms. For each term, we
obtained 1,000 articles and extract their adjectives. We se-means combining three independent visual features two by
two to get three methods of color-shape, shape-texture andcolor-texture respectively. First, we obtain the visual similarity of fashion goods and all visual metaphors candidates
based on single visual features. Then the two visual similarities are combined with the following formula. Sima andSimb are any two visual similarities in color, shape, and texture, respectively. The double visual similarity is determinedby two values, one is the average of Sima and Simb . The
second is the maximum value of Sima and Simb .
√Sima + Simb
Sim =max(Sima , Simb )2lect the adjective with highest frequency among adjectives

All visual metaphors were ranked by the similarity calcu-

shared by fashion goods and visual metaphor to calculate

lated from the above formula to obtain the visual metaphorssimilarity with product description. The model we used toof fashion goods based on the method of double visual simi-

calculate the text similarity is Japanese pre-learning BERT

larity.

model from Tohoku University [9]. Representative adjective

Method based on three visual similarity is to combine threeof metaphor word and product description are transformed

visual similarities to present all visual metaphors in one rank-

into text vectors, and their cosine similarity is calculated toing. Process of combination is shown in Figure 8. Firstly,obtain the conceptual similarity.

We output single visual similarity rankings based on color,

Metaphorical Appropriateness Computation

shape, texture. Then separate all metaphor words into three

An overview of the visual metaphor generation method isgroups: (a) visual metaphors that appear on all three rank-

shown in Figure 7. Metaphorical appropriateness of visual

ings, (b) visual metaphors that appear on two rankings, andmetaphors and fashion goods consists of two parts: visual(c) visual metaphors that appear on one ranking only. We

similarity and conceptual similarity. Based on diﬀerent treat-sort each group by the mean of the ranks of each metaphor

ments of color, shape, and texture similarities, there are dif-word.

ferent approach to obtain metaphorical appropriateness. We

5.

4. 3.

Evaluation

will illustrate three methods of visual metaphor generation
The manual-based evaluation consists of two aspects: theTable 5 Evaluation results of diﬀerent methods for appearance
aspectmethodsingle-sim

double-sim
three-simbest score avg score
color

6.4

4.4

shape

5.7

3.0

texture5.9

3.2

color&shape

6.4

4.2

color&texture

6.4

4.2

shape&texture

6.1

3.2

color&shape&texture

6.4

3.8

Figure 8 The process of visual metaphor generation based on
three visual featuresappearance aspect and the conceptual aspect. In appearance
aspect, we evaluate the metaphor generation methods basedon single similarity, double similarity, and three similarities
in Section 4. 3. For conceptual aspect, we examined the effect of the diﬀerent importance of conceptual similarity on

Figure 9 Color similarity is evaluated more than shape and texture

visual metaphor generation. We introduced a parameter α
to regulate the degree of visual and conceptual contribution

judging the validity of visual metaphor. As shown in Fig-

to the establishment of metaphors. As shown in the follow-

ure 9, the visual metaphor based on shape and texture anding equation, M is the metaphorical appropriateness, Simv

the visual metaphor based on a single visual similarity ofand Simc are the visual similarity and conceptual similar-

color are planet and chocolate millefeuille. We can observeity. We generated metaphors for each item based on three αthat the product has a typical round shape as well as planet,(α = 0.7, α = 0.8, α = 0.9) methods.

which proves that planet and product have a very convincing

1−α
M = Simαv SimcWe selected 20 items with distinctive features in color,shape, and texture from the test set of the visual metaphor

similarity in shape. However, the similarity in color is doubtful, because the impressions of planet in terms of color are
diverse. The less convincing color similarity between planetand commodities led workers to score the output low.

We take top 10 ranked

Table 6 shows performance of visual metaphors generated

metaphors generated by each method as the result of eachbased on diﬀerent importance degrees of conceptual similar-

method and present shuﬄed results to workers and asked

ity. It is concluded that the variation of α from 0.7 to 0.9

them to answer following two questions.

did not aﬀect the performance of visual metaphors in re-dataset for manual evaluation.

Look at the product image and the visual metaphor.

ﬂecting descriptions. We use an example shown in Figure 10

Would you say that this visual metaphor is valid for thisto explore the reason behind this result. We assume that

product image?

the workers interpreted the feature of “stylized design” as•

Look at the product image and the visual metaphor.

individuality and unexpectedness when they understood theWould you say that this visual metaphor is valid for thisdescription of this product. The design of product remind

product description?

people that it is a performance costume, so stage is the clos-

For each question, workers were asked to answer a number

est to the product in terms of function, but it does not reﬂect

from 1 to 10, which indicates 10 levels of satisfaction (1 being

individuality and unexpectedness. Although wedding veil is

the worst and 10 being the best). We averaged the scores

more unexpected and surprising than stage, its impression is

answered by the three workers as the ﬁnal rating value.

more considered as a symbol of tenderness than highlighting•

Table 5 shows results of appearance aspect. We can learn

individuality. While pearl shell, with its shiny appearance

that people are most concerned with color similarity whenand rarity, gives it a ﬂashy impression and ﬁts the individ-

Figure 12 Examples of catchphrase generation(“ta”, “no”, “na”, “i”) to build 4 queries. “Ta” is auxiliary
Figure 10 Visual metaphors obtained based on diﬀerent concep-

verb in Japanese, which is used behind verb. “No” is auxiliary that connects noun to the metaphor word, which can betual similarity weights

regarded as “of” in English. “I” is word used in Japanese to
Table 6 Evaluation results of diﬀerent methods for conceptual
aspectfollow an adjective. “Na” is auxiliary verb that connect adjective and metaphor word, which can be regarded as “ful”

α = 0.9 α = 0.8 α = 0.7
best score

6.4

6.4

6.3

avg score

3.2

3.2

3.2

at the end of an English adjective. We searched 1,000 resultsfor each of query, collected 25 to 110 rhetoric candidates foreach metaphor word. We obtained rhetoric with the highest
similarity to product description based on BERT. Then use
this rhetoric as ﬁrst word in rhetoric part of catchphrase. We
add a random cue character and [MASK] for mask estimation
to complete catchphrase.
Figure 12 shows examples of catchphrase generation forthree items. We took three metaphor words from the top 10of the visual metaphors generated for each product as the

Figure 11 Process of catchphrase generation

metaphorical part of the catchphrase, and then generated
rhetoric based on them to complete the catchphrase.

uality part of the product description, all three metaphorshave a positive impression in terms of appearance, but pearl
shell is the one that best highlights the unexpectedness.

6.

Catchphrase Generation based on

The ﬁrst example in Figure 12 is a pink and white knitted
sweater with a gentle touch. Catchphrases “thick colorful
cream” and “baked whitish roll cake” contain adjectives related to color because the color scheme of the product ishighlighted in the product description. In addition, “calmcomfortable bedding” focuses on the part of the product de-Visual Metaphor

scription that emphasizes the feel of the clothes, highlight-

The process of generating catchphrases is shown in Fig-

ing the softness of the product. In the second example inure 11. We complete the catchphrase by adding rhetoric in

Figure 12, the metaphor words are all animals, and they

front of the metaphor word. The ﬁrst step is to select theshare the common rhetoric “Energetic tender”. Since thecandidate most similar to the product description from theﬁrst rhetoric is the most similar rhetoric to the product de-common rhetorics of the metaphor word and form a phrase.

scription among rhetoric candidates of the metaphor word,

The second step is to add mask in front of this phrase to

the repeated word is output when the word is also present

generate the second rhetoric by BERT mask prediction toin the rhetoric candidates of other metaphor words. The

complete catchphrase.

third example is a pair of light teal shoes with stretchy ma-

We collect rhetorics candidates of metaphor words by Bing

terial and soft to wear. Sesame pudding-like shoes will makeweb search API. We build queries for each metaphor word topeople associate the material of the shoes with the taste of

get web articles and extract rhetorics from them. Query is

pudding, and the smooth texture of the shoe material on

built by preﬁxing metaphor word with a cue character. For

the product picture also has some similarity with pudding.

one metaphor word, We use 4 cue characters in Japanese

The other catchphrase, cafe au lait, not only expresses thesmooth and delicate texture of the shoes, but also has a similar color to the shoes, which is visually more persuasive. The[6]last catchphrase is the most surprising, because Sphinx cat
does have a soft body, fully reﬂecting the description of thematerial in the shoe description. Moreover, Sphinx cat indeed has a similar color to the shoes. The rhetoric “smooth

[7]
[8]

soft” further draws attention to the Sphinx’s consistent characteristics with the product. For this example, without themodiﬁer, it would have been diﬃcult to capture the similar-[9]

ity between Sphinx cat and the product, as they are hardly
identical in appearance alone. Catchphrases increase the volume of information that the visual metaphor can convey, so
we can say that the visual metaphor and catchphrases have[10]

a complementary eﬀect.

7.

Conclusion[11]

In this paper, we construct a visual metaphor dataset of
fashion goods manually. We summarize three metaphor establishment patterns of color, shape, and texture, based on
which we construct three feature extraction models to implement visual metaphor generation. Based on the concep-[12]tual similarity between visual metaphors and fashion goods,
we discuss the use of visual metaphors in generating catchphrases. There are two improvement points in this paper[13]

that are worth exploring in the future. One is to increase thenumber of evaluation participants to analyze performanceof visual metaphors more comprehensively. The other is to[14]

achieve a variety of results even for simple-looking products.

Acknowledgements[15]This work was supported by JSPS KAKENHI Grant Numbers JP21H03775, JP21H03774, JP21H03554, JP22H03905.

[16]

In this paper, we used “Rakuten Dataset” (https://rit.
rakuten.com/data_release/) provided by Rakuten Group,
Inc. via IDR Dataset Service of National Institute of Infor[17]matics.
References[1] Swee Hoon Ang and Elison Ai Ching Lim. The inﬂuenceof metaphors and product type on brand personality perceptions and attitudes. Journal of Advertising, 35:39–53,
2006.
[2] Julia Birke and Anoop Sarkar. A clustering approach fornearly unsupervised recognition of nonliteral language. In
Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics, pages
329–336, 2006.
[3] Sandy Bulmer and Margo Buchanan-Oliver. Visual rhetoricand global advertising imagery. Journal of Marketing Communications, 12:49–61, 2006.
[4] Shuo Cao, Huili Wang, and Xiaoxia Zou. The eﬀect of visual
structure of pictorial metaphors on advertisement attitudes.
International Journal of Marketing Studies, 10:60–72, 2018.
[5] Mathilde Caron, Piotr Bojanowski, Armand Joulin, andMatthijs Douze. Deep clustering for unsupervised learning

[18]

[19]

[20]

of visual features. In European Conference on Computer
Vision 2018, pages 139–156, 2018.
Giovanna Castellano and Gennaro Vessio. A deep learning
approach to clustering visual arts. International Journal ofComputer Vision, 130:2590–2605, 2022.
Forceville Charles. Pictorial metaphor in advertisements.
Metaphor and Symbolic Activity, 9:1–29, 1994.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li,
and Fei-Fei Li. ImageNet: A large-scale hierarchical image database. IEEE Conference on Computer Vision and
Pattern Recognition, 8.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina
Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the2019 Conference of the North American Chapter of the Association for Computational Linguistics, pages 4171–4186,
2019.
Se-Hoon Jeong. Visual metaphor in advertising: Is the
persuasive eﬀect attributable to visual argumentation ormetaphorical rhetoric? Journal of Marketing Communications, 14:59–73, 2008.
Youwen Kang, Zhida Sun, Sitong Wang, Zeyu Huang, Ziming Wu, and Xiaojuan Ma. MetaMap: Supporting visual metaphor ideation through multi-dimensional examplebased exploration. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–15,
2021.
Saif Mohammad, Ekaterina Shutova, and Peter Turney.
Metaphor as a medium for emotion: An empirical study. In
Proceedings of the Fifth Joint Conference on Lexical andComputational Semantics, pages 23–33, 2016.
Boujena Othman, Ulrich Isabelle, Piris Yolande, andChicheportiche Laëtitia. Using food pictorial metaphor in
the advertising of non-food brands: An exploratory investigation of consumer interpretation and aﬀective response.
Journal of Retailing and Consumer Services, 62, 2021.
Georgios Stampoulidis and Marianna Bolognesi. Bringing
metaphors back to the streets: A corpus-based study forthe identiﬁcation and interpretation of rhetorical ﬁgures instreet art. Visual Communication, 2019.
Gerard Steen, Lettie Dorst, J. Herrmann, Anna Kaal, Tina
Krennmayr, and Trijntje Pasma. A Method for Linguistic
Metaphor Identiﬁcation: From MIP to MIPVU. 2010.
Christian Szegedy, Sergey Ioﬀe, Vincent Vanhoucke, andAlexander A. Alemi. Inception-v4, Inception-ResNet andthe impact of residual connections on learning. In Proceedings of the Thirty-First AAAI Conference on Artiﬁcial Intelligence, pages 4278–4284, 2017.
Tsvetkov Yulia, Boytsov Leonid, Nyberg Eric, Gershman
Anatole, and Dyer Chris. Metaphor detection with crosslingual model transfer. In Proceedings of the 52nd Annual
Meeting of the Association for Computational Linguistics,
pages 248–258, 2014.
Dongyu Zhang, Minghao Zhang, Heting Zhang, Liang Yang,
and Hongfei Lin. MultiMET: A multimodal dataset for
metaphor understanding. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural
Language Processing, pages 3214–3225, 2021.
Huaizheng Zhang, Yong Luo, Qiming Ai, Yonggang Wen,
and Han Hu. Look, read and feel: Benchmarking ads understanding with multimodal multitask learning. In Proceedings of the 28th ACM International Conference on Multimedia, pages 430–438, 2020.
Haoxing Zhao and Xiaoyong Lin. A review of the eﬀect ofvisual metaphor on advertising response. In Proceedings ofthe 4th International Conference on Economy, pages 29–34,
2019.