JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 20191

Computation of sensory-affective relationships
depending onmaterial categories of pictorial stimuli
Shogo Okamoto, Member, IEEE, Kohta Wakamatsu, Shigeki Nakauchi, Jinhwan Kwon, Maki Sakamoto
Abstract—Exposure to stimuli gives rise to sensory and affective experiences. Computing the relationships between theseexperiences is instrumental to designing affectively appealing products and understanding human experiences. Hierarchical structuresof sensory and affective responses, which can be built through adjective rating tasks, are regarded as an effective means forexpressing relationships between sensory and affective responses. Naturally, these hierarchical structures depend on the type ofstimulus; however, so far, their dependencies on material categories have yet to be examined. We therefore investigated how
hierarchical structures of affective and sensory responses depend on seven material categories: fabric, leather, wood, paper, foliage,
stone, and glass. Each material category had 100 visual representations selected from the Flickr Material Database. Thirty-nine
participants were asked to rate 368 pictures across a set of materials. The questionnaire adjectives included visually- andhaptically-centered items although the stimuli were purely visual. We found that the structures differed substantially among the material
categories, although there were some commonalities. Particularly, the positions of polysemic or multimodal adjectives such as “light”
and “uncomfortable” in the hierarchy were highly dependent on the material category. For example, “light” has both physical(lightweight) and psychological (e.g., non-solemn, cheerful) meanings. For stone and glass (generally considered to be of heavy
weight), the psychological meanings were primarily considered. Conversely, the physical meanings were predominant for fabric,leather, wood, paper, and foliage, for which weight is a factor in judging quality. The present study helps interpret affective descriptors
whose meanings vary among types of stimuli.
Index Terms—Structural equation modeling, Flickr material database, Layered structure, Material perception, Affective responses

F

1

I NTRODUCTION

Humans experience changes in sensory and affective states
when exposed to products or sensory stimuli. The affective
responses, despite being linked to the perceived value of
products, do not seem to be directly linked with the physicalproperties of stimuli. Therefore, it is challenging to estimate how they change along with the physical propertiesof products. Hence, numerous researchers have attempted
to explain or predict affective responses by using sensory
adjectives, which are directly associated with the physicalproperties of the stimuli. To this end, many studies havecreated hierarchical models wherein scores are given to
sensory adjectives, which are then linked to scores of affective adjectives through unimodal or multimodal sensory
appraisals [1], [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12],[13], [14], [15].
These studies used their own methods of hierarchicalmodeling. Most studies sought to establish two-layered
models with sensory and affective adjectives at low and••••S. Okamoto is with the Department of Mechanical Systems Engineering,
Nagoya University, Japan.
K. Wakamatsu and S. Nakauchi are with the Department of Computer
Science and Engineering, Toyohashi University of Technology, Toyohashi,
Japan.
J. Kwon is with the Faculty of Education, Kyoto University of Education,
Kyoto, Japan.
M. Sakamoto is with Graduate School of Information Systems, The
University of Electro-Communications, Chofu, Japan.

Manuscript received June 11, 2019.

high layers, respectively [5], [7], [8], [11], [12]. The low
and high layers are typically connected by multivariate
regression analyses. Other studies used multi-layered models constructed using tasks other than the adjective ratingtask [3], [14], [15], or had experts in a given ﬁeld (e.g.,food or clothing industries) determine the structures of thelayers [1], [9], [10]. For example, Ueda [15] determined thehierarchy of 114 adjectives used for evaluating sounds on
the basis of their abstractness, complexity, objectiveness, anddifﬁculty, judged by 116 experimental participants. In [1],a hierarchical model to represent the sensory and affective
experiences while having coffee was made by means ofagreement among baristas. Multi-layered models allow forconnections among the affective variables and can express
more complex relationships among the affective responses;
hence, they represent affective responses more accurately.
For example, in [13], the mean correlation coefﬁcient between the observed and modeled ratings of ﬁve types
of affective descriptors was 0.67 for a two-layered model
whereas it was 0.75 in a four-layered model.
Fig. 1 provides an example of the layered structureof such a model. The sensory adjective scores are laidin the bottom layer and are directly linked to (i.e., they
psychophysically capture) the physical properties of stimuli. These sensory adjectives include “rough,” “warm,” etc.
The affective adjectives are instead typically located in the
middle and higher layers of the model and are explainedby adjectives in successively lower layers. The higher-layer
adjectives tend to pertain to hedonic or evaluative aspects,

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2019

Excitingboring

Generalspecial

RoughsmoothSurfaceroughness

͙Goodbad͙͙

Beautifulugly

Likedislike

Simplecomplex

Unevenflat

Bending
stiffness2

͙Comfort.uncomfort.

Affective
adjectives

Strangeusual

Hardsoft

Elasticity

͙͙

Stickyslippery

Sensory
adjectives

Thermalresistance

Coefficientof friction

Physicalpropertyof stimuli

Fig. 1. Layered structure of sensory and affective adjectives. Sensory
adjectives are directly linked with the physical properties of stimuli.
The affective adjectives in the middle to high layers are explained bysuccessively lower layers. Evaluative and hedonic aspects are generally
placed in the higher layers. This ﬁgure is consistent with [6].

whereas the middle-layer adjectives tend to describe theattributes of stimuli [1], [2], [4], [9], [14]. These layered
models provide semantic relationships between sensory andaffective responses, which can help in designing the affective values of products and interpreting the meanings ofconceptual values expressed by affective adjectives. Thusfar, these models have been leveraged for various products including beverages [1], clothing [10], car seats [8],leather [9], and wood [12].
In the present study, we establish hierarchical structuresof sensory and affective adjectives based on the scores assigned to these adjectives in a sensory appraisal task involving the presentation of pictures of seven material categories
including fabric, paper, wood, leather, foliage, glass, andstone. We then discuss the similarities and differences inthe sensory-affective relationships between these materials
using the hierarchical models of adjectives. As describedin the following two paragraphs, thus far, such sensoryaffective relationships have not been compared among material categories.
Thus far, using visual and haptic cues, sensory responses
to material surfaces and material classiﬁcation have been
extensively studied [16], [17], [18], [19], [20], [21]. Thesestudies showed that sensory responses depend on material
categories. For example, using computer graphics of several
material categories, Hiramatsu et al. [16] found that theimages of similar material categories formed clusters in
the perceptual space. Baumgartner et al. had similar results
using either visual or haptic material perception [19] where
the perceptual spaces made by the two sensory cues wereclose to each other. However, these studies did not focus on
the associations between perceptual and affective qualities.
In contrast, earlier studies on sensory and affective qualities did not focus on the differences between material categories. For example, Drewing et al. investigated the relationships between the affective qualities and dimensions of perceptual qualities using 47 types of objects to be touched [5].
However, they did not analyze their results from the view-

point of material categories, and the model was applied toa set of differing materials. Fujisaki et al. [12] compared
the relationships between sensory and affective responses tonatural woods according to the sensory cues available (i.e.,
when only tactile, visual, or auditory cues were available);they found that the relationships were similar across all
three modalities. They also found different sensory-affective
relationships between responses to wood and stone. However, to our knowledge, the sensory-affective relationships
have not been compared among several types of material
categories. Furthermore, it remains unclear how responses
to affective adjectives that are polysemic and indeﬁnite
depend on the material category. Understanding how these
relationships differ by material category could provide amore concrete scientiﬁc basis with regard to the industrial
design of products.
The potential contribution of the present study to industrial ﬁelds is as follows: In manufacturing, the affective
aspects experienced from materials are valued [22], and,thus far, systematic material-selection methods have beenstudied [23], [24], [25]. For example, Karana et al. proposeda method to help product designers to select materials
such that the affective attributes of products and materials
matched [23]. For customer–vendor communication about
products, customers tend to use indeﬁnite affective words
to convey their demands. In these situations, the designers
need to comprehend the meanings of affective descriptions.
The present study provides a computational approach forsuch purposes. Further, for designers of publication andweb pages, in addition to their own professional knowledge
and feelings, they may refer to the hierarchical models
established in this study as an aid with which to selectpictures that suit the contents of their texts. For example, theresults of the present study suggest that the impression of
comfort for stone is more psychological, and those appearing more beautiful are felt to be more comfortable, whereas
for other material categories, comfort is mostly determinedby the predicted mechanical comfort in touch. Although
the present study does not address image features, if they
are incorporated into our hierarchical models, the sensory
and affective impressions of pictures, which are not used
for training in the models, may be estimated. For example,
low-level image features can be linked with personal traitsof picture preferences [26] and perceptual qualities of object
surfaces [16].

2M ATERIALS AND M ETHODS

The study was approved by the institutional review boardof Toyohashi University of Technology.
2.1

Materials: Pictures of seven material categories

To investigate the relationships among sensory and affective
responses, one approach is to use a stimulus set, whereimage features pertaining to one type of sensory quality
is manipulated while controlling for the other parameters.
However, there are no general methods to manipulate
parameters corresponding to speciﬁc perceptual qualities
for natural images. Therefore, we used material pictures
provided by the Flickr Material Database (FMD) [27], forJOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 20193

OpaqueMatte

Transparent
Glossy

Moist

Dry
…

ModestFabric

Paper

Vivid

Glass

LeatherFig. 3. Visual analog scales for the questionnaire

Foliage

Wood

Stone

Fig. 2. Exemplar pictures of the seven materials categories. Those
actually used in the experiments are not shown because of copyrightconcerns, but they can be seen on the Flickr Material Database [27]website.

which multiple perceptual and affective properties vary
simultaneously among pictures. Because of the accessibilityof the database, we used image stimuli; nonetheless if thetask is extended to other cues, i.e., haptic sensations, thena certain level of consistency can be expected between the
results from visual and haptic tasks, although they may not
fully agree with each other [18], [28]. Especially, perceived
weight [29] may substantially differ among the visual andhaptic cues.
We used only colored pictures because color is part ofthe properties of natural and artiﬁcial materials, whereas
monochrome and colored pictures were compared in somestudies [20], [30]. We selected pictures that focused merelyon the materials and that give no indication of the object.
For example, some pictures were of glass cups, leather
bags, wooden eating utensils, etc.; we excluded these object
pictures. To retain a sufﬁcient number of pictures (i.e., one
hundred for each material category), we selected seven material categories: fabric, leather, foliage, glass, stone, paper,and wood1 . The stone category did not include gems. These
categories are typically used in the studies of visual material
perception [16], [17], [18], [20], [21].
The material categories of the FMD pictures analyzed
in the present study were easily identiﬁed with a successrate of 89% in our follow-up test, which involved sevenparticipants, different from those who participated in themain task2 . This value is similar to the rate reported in[30] where 91% of close-up and object images were correctlyclassiﬁed.
As shown in supplementary materials (Figs. A–C), therandomness of these pictures is suggested by the histogramsof participants’ subjective answers of perceptual and affec1. To establish a database, we also used pictures of water, plastic,
and metal. However, in the present study, we excluded these picturesfrom the analysis because the majority of pictures of metal and plastic
materials contained objects or products such as eating utensils, machine
tools, and coins for metal products, and did not focus on the surfaces.
Pictures of water are largely snapshots taken during motions such as
ﬂowing, falling, and dripping water. To reﬂect their affective properties,
we needed a set of adjectives different from those for other material
categories.
2. We excluded object images; however, the material classiﬁcation
rate was high. This is mainly because the tested images did not include
plastic materials which are generally confusing.

tive qualities to pictures. Their distributions are not fully
random, but the answers moderately gathered within limited ranges, with some pictures outlaid from the majority.
These suggest that pictures causing a variety of sensory andaffective qualities are included in each material category.
Fig. 2 provides exemplar pictures of each material. The
pictures were presented using a 32-inch LCD monitor with
1920 × 1080 pixels in a dark room.
2.2

Participants

Thirty-nine university students (30 men and 9 women, 20–
25 years old with the mean ± SD: 22.1 ± 1.2, native Japanese
speakers) who responded to an open advertisement participated in this study after providing their written informed
consent. They majored in engineering and were paid asdetermined by the institutional review board: 900 JPY/h.
They reported normal or corrected-to-normal visual acuity.
They had normal color vision as tested with Ishihara 38
color plates.
2.3

Procedures: Rating tasks of pictures

The participants rated each picture on a computer screen
using numerous visual analog scales, as shown in Fig. 3.
The extremities of these scales consisted of the two adjectives of each dyad, while the center of the scale, atwhich the sliders were initially positioned, was neutral.
The adjectives included rough–smooth, hard–soft, cold–warm,
sticky–slippery, moist–dry, matte–glossy, opaque–transparent,
uncomfortable–comfortable, modest (plain)–vivid (loud, bold),light–heavy, ugly–beautiful, cheap–luxury, and restless–relaxed(calm). All words were presented in participants’ native
language (i.e., Japanese)3 . Although some of the adjectives
are innately haptic, researchers have shown that visual
images can also give rise to haptic properties , e.g., [31], [32],[33], [34], [35], [36], [37]. For example, looking at glossy andrough material images activates the somatosensory area associated with tactile stimulation [37]. In earlier studies on visual material perception or classiﬁcation, haptic adjectives,
such as hard, elastic, fragile, and cold, were also used [16],[17], [18], [20], [21] considering the cross-modal connections
between visual and haptic sensations. The participants were
not provided any deﬁnition of the adjectives.
The descriptors selected were based on related studies sothat the number of descriptors would not be large. The ﬁrst
3. The corresponding Japanese words for the above list of adjectives are as follows: arai–nameraka, katai–yawarakai, tsumetai–atatakai,
nebatsuku–suberu, simetta–kawaita, tsuyanonai–tsuyanoaru, futomeina–
tomeina, fukai–kaiteki, jimina–hadena, karui–omoi, minikui–utsukushi,
yasuppoi–kokyuna, and ochitsukinonai–ochitsukinoaru, respectively. Furthermore, artiﬁcial–natural was rated aside the above thirteen adjective
dyads; however, it was not analyzed in the present study.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2019seven adjective dyads are frequently used in the study ofmaterial perception by visual cues (e.g. [12], [17], [20]). The
latter six dyads are used in the study of affective propertiesof materials (e.g. [7], [12], [14]). Nonetheless, the present
study does not aim to exhaustively cover various types ofaffective properties. It aims to demonstrate how middle andhigher levels of layered structures may differ among thedifferent types of materials.
For the appraisal of affective states of humans, nonverbal methods such as [38], [39] are also available. However, both affective and sensory aspects are covered in thepresent study. Non-verbal approaches covering these two
types of aspects do not exist, and we selected the method
based on descriptors.
Each participant rated 368 randomly selected picturesand different image sets. They included nearly equal numbers of pictures for each material category. On average,
six or seven participants rated one picture. They took abreak between every session containing 46 pictures andcompleted their assignments over 2 to 3 days to prevent
strain. Each session required approximately 30 min. In total
the experiment took approximately 5 hours.
2.4 Data analysis: Computation of hierarchical causal
structure of sensory and affective responses
We used the method in [13] to compute the hierarchical structure of affective and sensory adjective scores.
This method is based on structural equation modeling(SEM) [40], [41], which is suitable for analyzing causality
among multiple variables. As aforementioned in the introduction (second paragraph), the structure among the scoresof affective adjectives can be hypothesized and estimatedmore accurately in these models, than in two-layered models. The hypothesized models are statistically validated, as
will be discussed below. Another merit of the method in[13] for the present study is that a multi-layered structureis established solely based on an adjective rating task andmathematical computation whereas other methods makeuse of, or require, experts’ knowledge or another task fordetermining the hierarchy of adjectives (see the second
paragraph in the introduction). In general, multiple models
can statistically explain one observed dataset; however, themethod in [13] allows us to establish a concise model by
sequentially improving on the simplistic two-layered model.
That is, layers in the model with the lowest modelingaccuracy are continually developed until the entire structureis statistically acceptable.
The multi-layered modeling method in [13] yields a
structure where sensory adjectives are placed at the bottom
to explain the affective and evaluational adjectives in the
middle and higher layers as in Fig. 1.
The sensory responses were the following adjectives:rough–smooth, hard–soft, cold–warm, matte–glossy, opaque–transparent, sticky–slippery, and moist–dry. All the adjectives
are directly related to the physical properties of the material
surfaces and can be largely judged or estimated by theirappearances. In terms of non-visual adjectives i.e. hard-soft,
sticky-slippery, and cold-warm, according to [18], the ratingsin two tasks with either only haptic cues or visual cues,
were highly correlated, suggesting that these non-visual

4

Estimation
݁௪
error
ܽ௫௪
Estimation ݁
௫
error

w
Direct effectxܽ௭௪Correlation

ܽ௬௫

ܽ௭௫

w-z
y

z

Variable

Correlation

Fig. 4. Graphical expression of structural equation modeling. Unidirectional arcs and associated values are direct effects and their magnitudes, respectively. The effect values can be either positive or negative.
Nodes are the observed variables after normalization. e is the model
error, independent of the other variables and has a mean of zero.

properties are haptic-related visual properties, and can be
judged visually.
The remaining six adjectives (modest–vivid, cheap–luxury,
restless–relaxed, ugly–beautiful, uncomfortable–comfortable, andlight–heavy) were placed on the higher layers of the model.
Light–heavy is a haptic sensation, and visual judgment of thisquality requires longer response times than does judgmentof other visual qualities, such as glossiness and roughness [21]. Further, light–heavy tends not to be included in
rating tasks of perceptual qualities for visual images ofmaterials [17], [19], [20] because physical heaviness (i.e.,weight) is not judged solely from images of surface material. Hence, light–heavy was not included in the sensory
adjectives in the context of our visual task.
The layered structure created via SEM is graphicallyshown by using nodes and directed arcs in Fig. 4. The nodes
present the continuous variables after normalization (in thepresent study, the z -scores of the adjective scores), while
the directed arcs indicate a direct effect from one variable
to another. The strength of inﬂuence a is either positiveor negative. For example, the value of x is expressed as alinear summation of the effects of y , z , and model errorex (x = ayx y + azx z + ex ). The error follows a random
distribution and has a mean of zero. The inﬂuence values acorrespond to partial regression coefﬁcients of the multivariate regression analysis. x is also used to explain a variable in
the higher layer (i.e., w). Bidirectional arcs indicate that twoconnected variables (in the ﬁgure, y and z ) are signiﬁcantly
correlated.
SEM determines the coefﬁcients and statistical significance of the arcs in a layered structure, such that theobserved and estimated covariance matrices of the variables
resemble each other. To judge how accurately the hypothesized structure approximates the observed data, multiple
indices of model ﬁt are used [42], [43]. We used representative indices of the similarity between the observed andestimated covariance matrices, including the goodness-ofﬁt index (GF I ), comparative ﬁt index (CF I ), and χ2 . The
GF I and CF I range from 0 to 1; for statistically valid modeling, they should be greater than 0.90–0.95 [44], with the
maximum value of 1 indicating a complete match between
the two matrices. The χ2 value should, on the other hand, beJOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 20195

non-signiﬁcant and small (i.e., the p value should be greaterthan 0.05), indicating that the two covariance matrices are
not statistically different.
As well as the statistical validity, the semantic validityof the established model is also important for SEM, which
will be discussed in Secs. 4.1 and 4.2. The meanings of themodels are evaluated on the basis of common sense and theliterature available.
For each material category, we processed the adjectivescores as follows. First, we transformed the visual analoguescores for individual participants for each adjective into z scores for each material category such that their mean andstandard deviation are 0 and 1, respectively. The value foreach adjective dyad was bipolar: for example, if the value forrough–smooth was negative, it meant that the participant had
rated the material picture as smoother than average in thematerial category. The z -scores were then averaged among
the participants for each picture. We did not conduct any
outlier tests for this calculation. The covariance matrix of thethirteen adjective dyads was then computed and used in the
SEM. Sensory variables in the lower layer of the model wereallowed to correlate with each other due to the speciﬁcations
put in place. To conduct the SEM, we used the sem (version
3.1) package for R.

3

R ESULTS

For all material categories, the layered structures were successfully computed with p values greater than 0.05, and the
GFI and CFI values were mostly greater than 0.95. Fig. 5
shows the layered structure of each material. All the unidirectional arcs are statistically signiﬁcant (p < .05). For visualclarity, we do not show the correlations among the lowerlayer sensory adjectives. Instead, they are summarized inTables 1–7 in the Appendix. The adjectives that are notlinked with the others are also not displayed. Further, wehave omitted the graphical description of the errors. Instead,
we note the R2 value to indicate how accurately the value isestimated. R2 represents the square value of the correlation
coefﬁcient between the observed and estimated values. For
the same reason, each node displays only the adjective withthe positive value.
The materials showed several common features. First,
the higher layers tended to include restless, modest, andcheap, while the middle layer contained light, ugly, anduncomfortable. For the majority of materials, hard, matte, andrough inﬂuenced light, modest, and ugly, respectively. Despite
these commonalities, the layered structures mostly differedaccording to the material category. For some materials, cold,
opaque, and moist had no signiﬁcant impact on higher-layer
adjectives. For glass and stone, light was allocated to thehigher part of the structure, although it was just above thebottom perceptual layers for the other materials.
To investigate the similarities in participants’ responses
among the materials, we calculated the distances between
each of two material categories by using the following
formula:dab = tr[(X a − X b )T (X a − X b )](1)

where X a,b are the covariance matrices for materials a andb, which were in separate material categories (i.e., one of pa-

per, fabric, wood, leather, stone, glass, and foliage). This distance corresponds to the dissimilarity between two covariance matrices that deﬁne the structure among the observed
variables (adjectives). With a greater distance value between
two material categories, their variable structures are moredifferent from each other. These distances were then used
to compute the multidimensional scales of the covariancematrices [45], [46]. The covariance matrices of the sevenmaterial categories were arranged on a one-dimensional
scale, with the stress value being 0.015. Their locations areshown in Fig. 6. Five material categories were closely placed,
whereas glass and stone were at the two extremities. These
patterns indicated that participants’ responses were distinct
for glass and stone, whereas they were similar for the otherﬁve types of materials.
Further, to investigate how sensory and affective responses varied among the material categories and pictures,
we conducted a principal component analysis of the responses to pictures of all material categories for each set ofthe sensory and affective adjectives. As shown in Fig. 7 (a),the sensory responses to glass were substantially different
from those of the other materials. In contrast, the responses
to paper, fabric, and leather exhibited some similarities
that is, they were closely located. The responses to stonetended to be dissimilar to those of the others, but slightly
similar to the responses to wood. As shown in Fig. 7 (b),these trends were true for the affective responses as well.
Thus, the sensory and affective responses depended on thematerial categories, although they naturally varied withineach category because of the variety of pictures.

4

D ISCUSSION

4.1 Commonalities of fabric, leather, paper, wood, andfoliage
The shared link for all types of materials was that between
hard and light, with the other connections differing among
the materials. Still, as shown in Fig. 6, we found similar
questionnaire results for fabric, leather, foliage, wood, andpaper, and their layered structures were also rather similar. Note that these ﬁve types of material categories weregrouped into one, such that the following discussion is easy,
and we do not conclude that they have the same variable
structures. Fig. 8 shows a structure computed based on thequestionnaire responses from these ﬁve material categories.
This can be constructed as a common structure for thesematerial categories, and we discuss its characteristics andsemantic validity below.
Light and hard were strongly connected in a negative
manner. Light corresponded with weight, and the softer thesurfaces in the pictures appeared, the lighter the pictures
were scored. This is physically reasonable because soft objects tend to be of low density and are expected to be lightin weight, which agrees with [12], [21]. They are, however,judged to be independent, depending on experimental contexts [5].
Ugly was moderately connected with rough and cold.
Rougher surfaces were tended to be judged as uglier, as
were pictures with high cold scores; this suggests that uglyrefers not only to the apparent beauty of an object but
also haptic aspects. Impressions of beauty and thermal cues

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2019
Modest (.45)[Stone]6
Restless (.45)[Fabric]

Light (.42)

Uncomfort. (.64)

-.48

.22
.48

.20

-.36

Ugly (.73).59Modest (.39)

.60

-.25
.38

Restless (.45)

Cheap (.43).69  .95,
  1.00,
ଶ  30.0  28,
  0.36

.27
-.47

.33

Ugly (.52)

-.57

.18
.45

Rough

Sticky

Hard

-.17

Matte

.50

Light (.48)

Cheap (.38)
.24

.16

-.19

.42

Uncomfort. (.47).35

GFI = .94,
CFI = .99,
ଶ  35.5
  46ǡ  .26
.39

.34

.29

-.69

Moist

.20

Matte

Hard

Rough

[Paper][Foliage]

.41

.62

Moist

Restless (.53)

Modest (.71)
-.73

.29

-.55

.15

.48

Cheap (.42)

Modest (.32)-.33  .96,
  1.00,
ଶ 23.9  24,
  .47

.47

Cheap (.56)

Restless (.36)

.37

.55
-.20

.42

Uncomfort. (.70).17 .40-.13

  .95,
  1.00,
ଶ  37.5
  38,  .49

.73

.35
.25

.21

.62

.29

.18
.30

.58

-.61

.28

.17

.22
Hard

Opaque

Cold

-.14

Rough

Light (.35)-.16Sticky.25.39

Ugly (.71)

Cheap (.42)

.22

.69
.84

.31

.14

.37
.17

-.54

.14
Rough

.22

-.19

Ugly (.43)

Uncomfort. (.43).31Modest (.36)

.28

.63

.30

-.47

Uncomfort. (.65).25Cheap (.40)

.48

Moist

  .97,
  1.00ǡଶ  24.1
  37,   .95

-.38

-.19

.16

-.63

Cold

Light (.27)

Restless (.42)[Wood].24

OpaqueHard.28

Modest (.39)

Restless (.25)

.45

-.63

.30

Matte

[Glass]  .95,
  1.00,
ଶ  34.6
  44,   .84

.37

.16

.15

-.37

Rough

.17

Uncomfort. (.43)

Light (.42).16StickyUgly (.56)

.20

.15

Ugly (.38)

Light (.43)

.29

.18

.50

.35

.19

.28
StickyCold

Opaque

Hard

Rough

[Leather]

Sticky

Hard

Cold

Moist

Matte

Restless (.67)
-.13
-.46

  .96,
  1.00,
ଶ 29.5  33,   .64Modest (.36).45Light (.45)
.25

.16

-.87

Cheap (.55)

-.41
.34
.26

.12

.92

.33
-.25

.34
Uncomfort. (.68)
.71
-.39

Ugly (.45)
-.42
.19

Opaque

Rough

.20

.29

.17

Sticky

Hard

.14

.24

.34

.23
Cold

Moist

Matte

Fig. 5. Layered structure of affective and sensory responses to each of the seven types of materials. The adjectives that are not linked with anyothers are not shown. Values associated with the links are the strength of inﬂuences. Values in parentheses are R2 values. df means the degreeof freedom of the model.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2019

7

1.0

Hard
Cold

.5

Stone

0

Glass

Wood

Rough

StickyMatte
Paper
OpaqueFabric

Moist
Leather
Foliage

-.5
0
-.5
.5
1st principal component (Sensory) [47%]3rd principal component (Sensory) [11%]2nd principal component (Sensory) [27%](a) Sensory responses

Opaque
0

Hard

Paper
Leather
Wood
Glass
Stone
FabricFoliage
Cold
Matte
Rough

-.5
Sticky
Moist
0
-.5
.5
1st principal component (Sensory) [47%]Restless

.5

Light

Uncomfortable
Ugly Cheap
Fabric
Paper
Foliage
Leather
Glass
Stone
Wood

0

Modest0-.5.5
1st principal component (Affective) [43%]3rd principal component (Affective) [20%]2nd principal component (Affective) [25%](b) Affective responses
Light.5
Modest
Cheap
Paper
0

Wood

Foliage

FabricUgly Stone-.5

Leather
Uncomfortable
Restless

Glass

0
-.5
.5
1st principal component (Affective) [43%]Fig. 7. Materials plotted on the reduced principal-component space. Dots and ellipsoids are the centroids and standard deviations among thepictures, respectively. The values in brackets are the contribution ratios of the principal components.

Glass

Leather
FabricPaper
Wood
Foliage

Stone

20
0
-20
Distance between covariance matrices (a.u.)
Fig. 6. Loci of the material categories on a distance scale of theircovariance matrices.

are known to be synesthetic in the haptic perception ofmaterials [47] whereas it is not clear whether this ﬁndingﬁts into the present visual task. Furthermore, a study on
visual impression of furniture reported the relationships
among ugly, rough, modest, and uncomfortable [48], which iscomparable with the relationships in Fig. 8.
Uncomfortable was inﬂuenced by sticky, cold, and ugly,
which in turn were inﬂuenced by rough, cold, and matte.
These connections suggest that uncomfortable largely corresponded to the mechanical unpleasantness of touch [12],[49], [50], [51], [52], [53], [54], and that pictures that appeared
to stimulate the skin were judged as unpleasant.

Cheap was directly explained by ugly, uncomfortable, modest, light, and rough, which are reasonable features for lowprice surfaces. For example, woods that appeared light andmodest were judged to be cheap [12].
Restless is a complex characteristic comprising visual
beauty (ugly) and mechanical comfort (uncomfortable), whichmeans that relaxed (i.e., relieved) experiences are achievedby visually beautiful and mechanically comfortable stimuli.
Even when using visual or tactile cues only, feelings of relief(or relaxation) and comfort are strongly correlated [5], [12],[47].
We speculate that modest is conceptually similar to relieved (the opposite of restless) because of their strong association. In other words, modest pictures tend to be perceived
as relieving as well. Modest is directly linked with matte andhard and indirectly with opaque by way of ugly, which isconsistent with [55].
All these connections are reasonable and indicate that theaffective and evaluative relationships of the pictorial stimuli
were reasonably captured.
4.2

Distinctive characteristics of stone and glass

As shown in Fig. 6, the covariance matrices of stone andglass were substantially different from those of the othermaterials used in our study. Thus far, several studies have

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 20198

[Combination of 5 materials(fabric, leather, paper, wood, and foliage)]

Restless (.50).12.29Cheap (.51)
.32

-.69
Modest (.39)

.16

-.33

.08
.38

.26

.33
.32

.26

Uncomfort. (.67)

.15

.74

Ugly (.43)

Light (.66)

.23

.38

.19

.11

.07
.49

.30
-.81

.10

.12
-.09

OpaqueRoughSticky

Hard

Cold

Matte

Moist

Fig. 8. Layered structures computed for responses to fabric, leather, paper, wood, and foliage. GF I = .99, CF I = 1.00, and χ2 = 27.3 (df = 33,
p = .74). Dotted lines are weak but signiﬁcant connections.

analyzed the similarities in the subjective ratings of the perceptual features of images belonging to a variety of material
categories. These studies differed in the material categories,
questionnaires, panelist backgrounds, and analyses used;
hence, the conclusions are inconsistent. For example, insome studies, wood and stone exhibited similar responses
in questionnaires [17], [18] whereas in the other studies,
the responses for these categories were very different [12],[16], [20], [21]. The studies seem to weakly agree that the
responses to leather and fabric are similar. However, themajority of these studies exhibited dissimilarities between
stone and glass [16], [17], [20], [21]4 , which agrees with
our similarity analysis among the material categories, as inFigs. 6 and 7.
In the structure for stone, unlike the other materials, lightand uncomfortable were placed in the higher layers. Light
was directly inﬂuenced by hard, ugly, and restless. This suggests that light carried the psychological meanings (“gentle,”“delicate,” “cheerful,” “non-solemn,” and “non-serious”) as
well as the physical one (i.e., “lightweight”) [56]. Visualugliness might have imbued a picture with non-delicate andnon-gentle (heavy) impressions, whereas restless pictures
may have seemed cheerful (light). For the other materials,
we speculate that light largely meant “lightweight” becauselight was almost solely inﬂuenced by hard as in Fig. 8. In
contrast, stones are usually heavy and less distinctive interms of their weight. Hence, the psychological meanings
might have become more salient for stone.
For the majority of material categories, pictures thatlooked smooth, soft, and warm were rated as comfortable.
As noted above, uncomfortable seemed to be felt as the—strength of mechanical stimuli on the skin or body—that
is, mechanical unpleasantness. However, for stone, ugly
and restless stimuli were strongly related with uncomfortable, suggesting that uncomfortable also reﬂects visual un4. Glass was not used in [18].

pleasantness. Visual aesthetic value might have been moreinﬂuential for the impression of comfort when comparedto physical pleasantness because most stone objects are
haptically uncomfortable due to their surface roughness,hardness, and coldness.
The adjective structure for glass also differed from thoseof the majority of other materials. Similar to the structureof stone, light was placed in the highest position. Light was
linked with hard, cheap, and modest, suggesting that it carriedboth physical and psychological meanings. The direct negative effect of modest on light suggests that modest glasses
were felt as serious and solemn, which are the opposite
meanings of light in the psychological sense. Furthermore,
the direct effect of cheap on light suggests that cheap glasses
were felt as non-serious (i.e., light). The effect of cheap tolight was also rather complicated. Cheap had a direct positiveinﬂuence on light, but also an indirect negative inﬂuence via
modest. The affective responses in the higher layers of thestructure appear to be underlaid by complicated causality.
It also bears noting that the R2 value of light was merely.35, indicating that further adjectives are necessary to fully
describe this polysemic concept. For example, adjectives
related to artistic values of glass might be helpful for explaining the responses to light.
Light in the structure for wood appears exceptional. Light
was placed in the highest layer for wood, unlike for fabric,
paper, leather, and foliage, although these ﬁve types ofmaterials exhibited similar covariance matrices of adjectivescores. For wood, light might be an irregular case because
its R2 value was the smallest of all material categories. In
other words, the scores for light could not be well explainedby the other adjectives. For the quality judgment of wood,
attributes such as sturdiness, newness, and cleanliness aresubstantial factors [12]; however, these adjectives were notused in the present study. Both the appearance and impression of wood changes substantially with its age. For
example, new wood looks clean and the surface of old wood

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2019

does not look sturdy. These attributes might be related withimpressions of delicate and solemn, which are both includedin the meanings of heavy i.e. the opposite of light. Age might
be an important factor for evaluating wood; however, this
attribute was not included in our study, which might haveresulted in the low R2 values for light.
4.3

Dependence of layered structures on materials

Our main focus in this study was whether the semantic
structures of sensory and affective responses were different or similar among the different material categories. The
results in section 4.1 indicated that fabric, leather, paper,
wood, and foliage exhibited similar structures, while theﬁndings in section 4.2 suggest that glass and stone exhibited
different structures.
The polysemic nature of affective adjectives might beresponsible for the variations in the structures of affective
responses. Some affective adjectives have both physical andpsychological meanings and the salience of these meanings
might depend on the material categories.
For example, as mentioned in section 4.2, for typicallyheavy materials such as stone, participants seemed to havefocused on the psychological meanings of the adjective pairlight–heavy as well as the physical meanings.
For fabric, paper, leather, wood, and foliage, uncomfortable corresponded to the mechanical unpleasantness. In
contrast, for stone, uncomfortable was affected by restless andugly, indicating that the visual unpleasantness was more
activated. We might attribute this ﬁnding to the fact that thetactile comfort of stones is not usually discussed because
stone is generally uncomfortable due to its harsh and hardsurface. As such, for stone, comfort seemed to be based
primarily on its visual properties rather than its visuallyestimated, tactile properties.
Taken together, the results above suggest that thesalience of the meanings of polysemic adjectives vary withthe material category involved, which in turn, explains thedifferences in the hierarchical structures of adjectives. This
resembles the phenomenon where the meanings of polysemic words are automatically selected in speciﬁc contexts
while reading [57], [58], [59]. Lexical processing is activatedthrough pictures [60], [61], and thus the pictorial stimuli
used in the present study might have inﬂuenced the comprehension of polysemic affective adjectives.
In terms of the sensory adjectives, some distinctions werealso observed among the different material categories. Cold,moist, or opaque were not signiﬁcantly linked with the otheradjectives for some material categories, probably because
they do not represent the characteristics of these materials.
For example, most glass materials are glossy, meaning thatmoist and matte are not appropriate words to express theirdifferences.
4.4

Limitations of the study and future issues

Some pictorial characteristics that we did not consider in theexperiment could have weakened the accuracy of the model.
The material pictures comprised shapes, colors, and surface
patterns, in addition to their material characteristics. Shapehas been shown to inﬂuence abstract or affective impressions [62], [63], [64], [65], [66], [67], while the color of the9

pictures is known to inﬂuence judgments of warmth [34],[35], [36], [68], roughness, and softness [34], [69]. Colors arealso linked with some affective features [17], [20], [70]. The
removal of color (usage of gray images) may be an effective
method for excluding the effects of color, especially whengray images do not deter material classiﬁcation ratio [30].
These features, including shapes, surface patterns, and colors, are intrinsic properties of materials and we consider thatthey should not be fully controlled for.
As mentioned in Sec. 2.1, we did not use metal andplastic materials because many pictures in FMD are notclose-up pictures. Also, FMD does not include ceramic andrubber, which are sometimes used in the study of material
perception [16], [20]. In future, these materials will be tested
with new pictures to complete our study.
Similarly, if we use different adjective words in the ratingtasks, the similarities among the seven types of material
categories may change. For example, we did not include
natural-artiﬁcial or live-dead, for which foliage may evoke
unique response. Further, as aforementioned in Sec. 4.2, newold seems an important property for the perception of wood.
Hence, we should note that the similarities of the sensoryaffective relationships appear different, depending on thedescriptive word sets used, in the rating tasks.
The layered structures computed in the present study
indicate the semantic relationships of adjective ratings, andtherefore are different from the actual neural systems involved. Information processing in the human affective andhedonic systems is ﬂexible, and thus may differ dependingon material category. The SEM method is a linear approximation, whereas human perceptual and affective linkages
are likely to be nonlinear [71]. Additionally, SEM does
not account for recurrent structures. Hence, SEM might
not perfectly express the actual neural processing of therelationship between sensory and affective responses. In
addition, SEM does not provide a deﬁnite conclusion on
the relationships of observed variables; the structures tested
in this study by using SEM are but some of the statisticallypossible structures.
In the present study, still pictures were tested mainly
because of the availability of FMD images. Previous studies
have indicated that moving and interactive images enhancesensory properties such as the softness and glossiness ofmaterials [72], [73]. Hence, in the future, our approach
will be coupled with videos focusing on materials. Further,
the perceptual dimensions of visual and haptic material
perception are similar to each other, except for the properties
unique to visual perception such as colorfulness and glossiness [12], [18], [74], [75]. However, in terms of the affective
responses, the similarity between the visual and haptic cues
has yet to be established. Hence, the ﬁndings in the present
study are limited to pictorial stimuli.

5

C ONCLUSION

In the present study, we had participants rate pictures ofseven material categories using seven sensory and six affective adjectives. Through SEM, we created a layered structurethat semantically and statistically explained the observed
adjective scores for each material category. Participants’
responses for ﬁve material categories—fabric, paper, wood,

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 201910

leather, and foliage—were similar. For these materials, theresponses to relieved–restless, vivid–modest, and luxury–cheap
were in the higher layer of the structure and were inﬂuenced
by the other affective and sensory responses. In contrast,
for stone, light–heavy and comfortable–uncomfortable appearedin the high layers, while for glass, light–heavy occupied
the highest position in the structure. We speculate that therelative importance of the physical (weight) and psychological meanings of light–heavy rely on the material category.
For some materials, light is more physical, while for othermaterials, it is more psychological. Stone and glass are bothheavy in weight, which perhaps makes the physical meaning of light less important or salient when evaluating their
quality. Further, comfortable comprises a predicted mechanical pleasantness and visual beauty, and the precise meaningthat arose may have depended on the materials involved.
For stone, which is generally rough and unpleasant on
the skin, comfortable might have been interpreted mainly
as an aspect of visual beauty. The material-dependency ofaffective responses found through our adjective-rating tasks
suggests the mutability of the human affective system.

C ORRELATION COEFFICIENTS AMONG SENSORY RE -TABLE 1
Correlation coefﬁcients among sensory variables for fabric. Signiﬁcantvalues (p < .05)) are listed.
Rough
.15
-.23
.45
-.23
.63
.23

Hard
.45
.19
.40
-.21
-.19

Cold

Sticky

Moist

Matte

-.39
-.49

.35

.42
-.45
-.42

.20
.33

TABLE 2
Correlation coefﬁcients among sensory variables for paper. Signiﬁcantvalues (p < .05)) are listed.

Hard
Cold
StickyMoist
Matte
Opaque

Rough
.19

Hard
.37

.58
.33
.17

Cold
-.24

Sticky
.23

-.24

Moist

Matte

-.33
-.40

.52

TABLE 3
Correlation coefﬁcients among sensory variables for leather. Signiﬁcantvalues (p < .05)) are listed.

Hard
Cold
StickyMoist
Matte
OpaqueHard
Cold
StickyMoist
Matte
Opaque

Rough
.30
.23
.40
-.24
.49

Hard
.58
.17
-.18

Cold

Sticky

Moist

Matte

-.23
-.26

.30

-.17
-.19

.34

ACKNOWLEDGMENTS
This study was partly supported by JSPS Kakenhi(15H05922 and 15H05923) and (17K20002).

Rough

Hard
-

.41
-.38

-.52
-.62

Cold

Sticky

Moist

.22

-

Matte

-

TABLE 5
Correlation coefﬁcients among sensory variables for glass. Signiﬁcantvalues (p < .05)) are listed.

Hard
Cold
StickyMoist
Matte
Opaque

Rough

Hard
-

.44

-.47
-.60
-.28

Cold

Sticky

Moist

Matte

.64
.36

.44
.45
.18

-.23

.52

R EFERENCES[1]

SPONSES

Hard
Cold
StickyMoist
Matte
OpaqueTABLE 4
Correlation coefﬁcients among sensory variables for stone. Signiﬁcantvalues (p < .05)) are listed.

T. Nishino, M. Nagamachi, and M. Sakawa, “Acquisition of kansei
decision rules of coffee ﬂavor using rough set method,” Kansei
Engineering International, vol. 5, no. 4, pp. 41–50, 2006.
[2] X. Chen, C. J. Barnes, T. H. C. Childs, B. Henson, and F. Shao,“Materials’ tactile testing and characterization for consumer products’ affective packaging design,” Materials and Design, vol. 30, pp.
4299–4310, 2009.
[3] S. Okamoto, H. Nagano, K. Kidoma, and Y. Yamada, “Speciﬁcation of individuality in causal relationships among texturerelated attributes, emotions, and preferences,” International Journalof Affective Engineering, vol. 15, no. 1, pp. 11–19, 2016.
[4] K. Kidoma, S. Okamoto, H. Nagano, and Y. Yamada, “Graphicalmodeling method of texture-related affective and perceptual responses,” International Journal of Affective Engineering, vol. 16, no. 1,
pp. 27–36, 2017.
[5] K. Drewing, C. Weyel, H. Celebi, and D. Kaya, “Systematic
relations between affective and sensory material dimensions intouch,” IEEE Transactions on Haptics, pp. 1–1, 2018.
[6] H. Nagano, S. Okamoto, and Y. Yamada, “Modeling semanticallymultilayered affective and psychophysical responses toward tactile textures,” IEEE Transactions on Haptics, vol. 11, no. 4, pp. 568–
578, 2018.
[7] S. Guest, J. M. Dessirier, A. Mehrabyan, F. McGlone, G. Essick,G. Gescheider, A. Fontana, R. Xiong, R. Ackerley, and K. Blot,“The development and validation of sensory and emotional scalesof touch perception,” Attention, Perception & Psychophysics, vol. 73,
pp. 531–550, 2011.
[8] T. Matsuoka, H. Kanai, H. Tsuji, T. Shinya, and T. Nishimatsu,“Predicting texture image of covering fabric for car seat by physical properties,” Journal of Textile Engineering, vol. 54, no. 3, pp.
63–74, 2008.
[9] I. H. M. Hashim, S. Kumamoto, K. Takemura, T. Maeno, S. Okuda,
and Y. Mori, “Tactile evaluation feedback system for multi-layered
structure inspired by human tactile perception mechanism,” Sensors, vol. 17, p. 2601, 2017.
[10] S. Kawabata and M. Niwa, “Objective measurement of fabrichand,” in Modern Textile Characterization Methods, M. Raheel, Ed.
CRC Press, 1996, pp. 329–354.
[11] R. Ackerley, K. Saar, F. McGlone, and H. B. Wasling, “Quantifying the sensory and emotional perception of touch: Differences
between glabrous and hairy skin,” Frontiers in Behavioral Neuroscience, vol. 8, no. 34, 2014.
[12] W. Fujisaki, M. Tokita, and K. Kariya, “Perception of the material
properties of wood based on vision, audition, and touch,” Vision
Research, vol. 109, pp. 185–200, 2015.
[13] S. Okamoto, H. Kojima, A. Yamagishi, K. Kato, and A. Tamada,“Layered-modeling of affective and sensory experiences using
structural equation modeling: Touch experiences of plastic sur-

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 201911

TABLE 6
Correlation coefﬁcients among sensory variables for foliage. Signiﬁcantvalues (p < .05)) are listed.

Hard
Cold
StickyMoist
Matte
Opaque

Rough
-.16

Cold

.19

Hard
.35
-.22

.64

-.22

-.17

Sticky

Moist

Matte

.32

-

TABLE 7
Correlation coefﬁcients among sensory variables for wood. Signiﬁcantvalues (p < .05)) are listed.
Rough
Hard
Cold
StickyMoist
Matte
Opaque

.38
-.34
.48
.24

Hard
.22
-.30
-.37
.42

Cold

Sticky

Moist

Matte

-.24
-.40

.30

.15
.22

faces as an example,” IEEE Transactions on Affective Computing,2018.
[14] H. Nagano, S. Okamoto, and Y. Yamada, “Semantically layered
structure of tactile textures,” in Haptics: Neuroscience, Devices,
Modeling, and Applications, ser. Lecture Notes in Computer Science,
M. Auvray and C. Duriez, Eds. Springer, 2014, vol. 8618, pp. 3–9.
[15] K. Ueda, “A hierarchical structure for adjectives describing timbre,” The Journal of the Acoustical Society of America, vol. 100, no. 4,
p. 2751, 1996.
[16] C. Hiramatsu, N. Goda, and H. Komatsu, “Transformation fromimage-based to perceptual representation of materials along thehuman ventral visual pathway,” Neuroimage, vol. 57, pp. 482–494,
2011.
[17] R. W. Fleming, C. Wiebeland, and K. Gegenfurtner, “Perceptual
qualities and material classes,” Journal of Vision, vol. 13, no. 9, p. 9,
2013.
[18] E. Baumgartner, C. B. Wiebel, and K. R. Gegenfurtner, “Visualand haptic representations of material properties,” Multisensory
Research, vol. 26, pp. 429–455, 2013.
[19] ——, “A comparison of haptic material perception in blind andsighted individuals,” Vision Research, vol. 115, pp. 238 – 245, 2015,
perception of Material Properties (Part II).
[20] M. Tanaka and T. Horiuchi, “Investigating perceptual qualitiesof static surface appearance using real materials and displayed
images,” Vision Research, vol. 115, pp. 246 – 258, 2015.
[21] T. Nagai, T. Matsushima, K. Koida, Y. Tani, M. Kitazaki, and
S. Nakauchi, “Temporal properties of material categorization andmaterial rating: visual vs non-visual material features,” Vision
Research, vol. 115, pp. 259–270, 2015.
[22] E. Karana, P. Hekkert, and P. Kandachar, “Materials experience:descriptive categories in material appraisal,” in Proceedings of International Conference Tools and Methods in Competitive Engineering,
2008, pp. 399–412.
[23] ——, “A tool for meaning driven materials selection,” Materials &Design, vol. 31, no. 6, pp. 2932–2941, 2010.
[24] M. Wiberg, “Methodology for materiality: interaction design research through a material lens,” Personal and Ubiquitous Computing,vol. 18, no. 3, pp. 637–649, 2014.
[25] E. Giaccardi and E. Karana, “Foundations of materials experience:
An approach for HCI,” in Proceedings of SIGCHI Conference on
Human Factors in Computing Systems, 2015, pp. 2447–2456.
[26] C. Segalin, A. Perina, M. Cristani, and A. Vinciarelli, “The pictures
we like are our image: Continuous mapping of favorite pictures
into self-assessed and attributed personality traits,” IEEE Transactions on Affective Computing, vol. 8, no. 2, pp. 268–285, 2017.
[27] “Flickr material database,” 2018. [Online]. Available: https://people.csail.mit.edu/celiu/CVPR2010/FMD/[28] C. Spence, “Crossmodal correspondences: A tutorial review,” Attention, Perception, & Psychophysics, vol. 73, no. 4, pp. 971–995, 2011.

[29] P. Walker, B. J. Francis, and L. Walker, “The brightness-weightillusion. Darker objects look heavier but feel lighter,” Experimental
Psychology, vol. 57, pp. 462–469, 2010.
[30] L. Sharan, R. Rosenholtz, and E. H. Adelson, “Accuracy and speedof material categorization in real-world images,” Journal of Vision,
vol. 14, no. 9, p. 12, 08 2014.
[31] R. W. Fleming, “Material perception,” Annual Review of Visual
Science, vol. 3, pp. 365–388, 2017.
[32] E. H. Adelson, “On seeing stuff: the perception of materials byhumans and machines,” Proceedings of the SPIE, Human Vision andElectronic Imaging VI, vol. 4299, 2001.
[33] H. Nagano, S. Okamoto, and Y. Yamada, “Haptic invitation of textures: Perceptually prominent properties of materials determine
human touch motions,” IEEE Transactions on Haptics, vol. 7, no. 3,
pp. 345–355, 2014.
[34] V. U. Ludwig and J. Simner, “What colour does that feel? tactile
visual mapping and the development of crossmodality,” Cortex,
no. 49, pp. 1089–1099, 2013.
[35] L. Wastiels, H. N. J. Schifferstein, A. Heylighen, and I. Wouters,“Red or rough, what makes materials warmer?” Materials andDesign, vol. 42, pp. 441–449, 2012.
[36] H.-N. Ho, D. Iwai, Y. Yoshikawa, J. Watanabe, and S. Nishida,“Combining colour and temperature: A blue object is more likely
to be judged as warm than a red object,” Scientiﬁc Reports, vol. 4,
p. 5527, 2014.
[37] H.-C. Sun, A. E. Welchman, D. H. F. Chang, and M. Di Luca,“Look but don’t touch: Visual cues to surface structure drive
somatosensory cortex,” Neuroimage, vol. 128, pp. 353–361, 2016.
[38] K. Isbister, K. Höök, J. Laaksolahti, and M. Sharp, “The sensual evaluation instrument: Developing a trans-cultural self-reportmeasure of affect,” International Journal of Human-Computer Studies,vol. 65, pp. 315–328, 2007.
[39] P. M. A. Desmet, “Measuring emotions: development and application of an instrument to measure emotional responses to
products,” M. A. Blythe, A. F. Monk, K. Overbeeke, and P. C.
Wright, Eds. Springer, 2004, pp. 111–123.
[40] P. M. Bentler, “Linear systems with multiple levels and types oflatent variables,” in Systems under indirect observation: Causality,
structure, prediction, K. G. Joreskog and H. Wold, Eds., 1982, pp.
101–130.
[41] ——, “Structural modeling and psychometrika: An historical perspective on growth and achivements,” Psychometrika, vol. 51, pp.
35–51, 1982.
[42] S. A. Mulaik, L. R. James, J. V. Alstine, N. Bennett, S. Lind, andC. D. Stilwell, “Evaluation of goodness-of-ﬁt indices for structural
equation models,” Psychological Bulletin, vol. 105, no. 3, pp. 430–
445, 1989.
[43] D. Hooper, J. Coughlan, and M. Mullen, “Structural equation modelling: Guidelines for determining model ﬁt,” Electronic Journal ofBusiness Research Methods, vol. 6, no. 1, pp. 53–60, 2008.
[44] R. P. McDonald and M.-H. R. Ho, “Principles and practice in reporting structural equation analyses,” Psychological Methods, vol. 7,
no. 1, pp. 64–82, 2002.
[45] J. B. Kruskal, “Multidimensional scaling by optimizing goodnessof ﬁt to a nonmetric hypothesis,” Psychometrika, vol. 29, pp. 1–27,
1964.
[46] ——, “Nonmetric multidimensional scaling: A numericalmethod,” Psychometrika, vol. 29, pp. 115–129, 1964.
[47] R. Etzi, C. Spence, M. Zampini, and A. Gallace, “When sandpaperis ‘kiki’ and satin is ‘bouba’: an exploration of the associations
between words, emotional states, and the tactile attributes ofeveryday materials,” Multisensory Research, vol. 29, no. 1–3, pp.
133–155, 2016.
[48] Y.-T. Chen and M.-C. Chuang, “The difference of imagery perception between visual and tactile sensation,” International Journal of
Affective Engineering, vol. 14, no. 3, pp. 233–240, 2014.
[49] G. K. Essick, F. McGlone, C. Dancer, D. Fabricant, Y. Ragin,
N. Phillips, T. Jones, and S. Guest, “Quantitative assessment ofpleasant touch,” Neuroscience and Biobehavioral Reviews, vol. 34, pp.
192–203, 2010.
[50] A. Klöcker, C. Arnould, M. Penta, and J.-L. Thonnard, “Rasch-built
measure of pleasant touch through active ﬁngertip explorations,”
Frontiers in Neurorobotics, vol. 6, no. 5, 2012.
[51] S. R. Bhatta, K. Tiippana, K. Vahtikari, M. Hughes, and M. Kyttä,“Sensory and emotional perception of wooden surfaces throughﬁngertip touch,” Frontiers in Psychology, vol. 8, p. 367, 2017.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2019[52] H. Nagano, S. Okamoto, and Y. Yamada, “Visual and sensory
properties of textures that appeal to human touch,” International
Journal of Affective Engineering, vol. 12, pp. 375–384, 2013.
[53] M. Shitara, H. Yoshida, M. Kamijo, G. Fujimaki, and H. Yamaguchi, “Hand movements used to assess the comfortability andlikability of wood,” International Journal of Affective Engineering,
vol. 17, no. 1, pp. 49–56, 2017.
[54] A. Klöcker, M. Wiertlewski, V. Th’eate, V. Hayward, and J.-L.
Thonnard, “Physical factors inﬂuencing pleasant touch during
tactile exploration,” Plos one, vol. 8, no. 11, p. e79085, 2013.
[55] E. Karana, B. Barati, V. Rognoli, and A. Z. van der Laan, “Material
driven design (MDD): A method to design for material experiences,” International Journal of Design, vol. 9, no. 2, 2015.
[56] “Oxford dictionaries,” 2018. [Online]. Available: https://en.
oxforddictionaries.com/[57] R. Frost and S. Bentin, “Processing phonological and semantic
ambiguity: Evidence from semantic priming at different soas,”
Journal of Experimental Psychology: Learning Memory and Cognition,
vol. 18, no. 1, pp. 58–68, 1992.
[58] J. L. McClelland, The case for interactionism in language processing,ser. Attention and performance, 1987, vol. 12, pp. 3–36.
[59] M. Hisano, “Advances and issues in lexical ambiguity research,”
Bulletin of the Faculty of Education, University of Tokyo, vol. 33, pp.
107–115, 1993.
[60] T. Marinis, “Cross-modal priming in bilingual sentence processing,” Bilingualism: Language and Cognition, vol. 21, no. 3, pp. 456–
461, 2018.
[61] T. H. Carr, C. McCauley, R. D. Sperber, and C. M. Parmelee,“Words, pictures, and priming: on semantic activation, conscious
identiﬁcation, and the automaticity of information processing,”
Journal of Experimental Psychology: Human Perception and Performance, vol. 8, no. 6, pp. 757–777, 1982.
[62] W. Köhler, Gestalt psychology (2nd ed.). Oxford, 1947.
[63] B. Lyman, “Representation of complex emotional and abstract
meanings by simple forms,” Perceptual and Motor Skills, vol. 49,
no. 3, pp. 839–842, 1979.
[64] C. H. Liu and J. Kennedy, “Symbolic forms and cognition,” Psyke& Logos, vol. 14, pp. 441–456, 1993.
[65] G. L. Collier, “Affective synesthesia: Extracting emotion spacefrom simple perceptual stimuli,” Motivation and Emotion, vol. 20,
no. 1, pp. 1–32, 1996.
[66] A. Salgado Montejo, J. Alvarado, C. Velasco, C. Salgado, K. Hasse,and C. Spence, “The sweetest thing: the inﬂuence of angularity,
symmetry, and the number of elements on shape-valence andshape-taste matches,” Frontiers in Psychology, vol. 6, p. 1382,
2015. [Online]. Available: https://www.frontiersin.org/article/
10.3389/fpsyg.2015.01382[67] K. Taguchi, M. Hashimoto, K. Tobitani, and N. Nagata, “An estimation method of human impression factors for objects from their
3D shapes using a deep neural network,” in IS&T International
Symposium on Electronic Imagin, 2018, p. 194.
[68] M. Ziat, C. A. Balcer, A. Shirtz, and T. Rolison, A century later, thehue-heat hypothesis: Does color truly affect temperature perception?, ser.
Lecture Notes in Computer Science. Springer, 2016, vol. 9774, pp.
273–280.
[69] J. Simner and V. U. Ludwig, “The color of touch: A case of tactilevisual synaesthesia,” Neurocase, vol. 18, no. 2, pp. 167–180, 2012.
[70] S. Kobayashi, “The aim and method of the color image scale,”Color Research & Application, vol. 6, no. 2, pp. 93–107, 1981.
[71] F. Camargo and B. Henson, “Aligning affective responses withfabric features of vehicle seat: An approach using the rasch
measurement model,” International Journal of Affective Engineering,
vol. 14, no. 3, pp. 193–202, 2015.
[72] B. Petreca, B.-B. N., S. Baurley, P. Watkins, and D. Atkinson,“An embodiment perspective of affective touch behaviour inexperiencing digital textiles,” in Proceedings of Humaine Association
Conference on Affective Computing and Intelligent Interaction, 2013,
pp. 770–775.
[73] Y. Sakano and H. Ando, “Effects of head motion and stereo
viewing on perceived glossiness,,” Journal of Vision, vol. 15, no. 9,
p. 15, 2010.
[74] S. Okamoto, H. Nagano, and Y. Yamada, “Psychophysical dimensions of tactile perception of textures,” IEEE Transactions on
Haptics, vol. 6, no. 1, pp. 81–93, 2013.
[75] Y. Vardar, C. Wallraven, and K. J. Kuchenbecker, “Fingertip interaction metrics correlate with visual and haptic perception of real

12

surfaces,” in Proceedings of IEEE World Haptics Conference, 2019, pp.
395–400.

Shogo Okamoto acquired his Ph.D. in informatics from Tohoku University in 2010. Currently, heis an associate professor at Nagoya University,
Japan. His research interests include haptics,
assistive systems, man-machine interfaces, andaffective engineering.

Kohta Wakamatsu received a M.S. degree from
Toyohashi University of Technology in 2019. Currently, he is a Ph.D. candidate.

Shigeki Nakauchi received a Ph.D. in engineering from Toyohashi University of Technology in
1993. He is currently a professor at Toyohashi
University of Technology, Toyohashi, Japan.

Jinhwan Kwon received a Ph.D. in sciencefrom Tokyo Institute of Technology in 2015. Heworked as a researcher at the Tokyo Instituteof Technology and the University of ElectroCommunications in 2017. He is currently working as a lecturer at Kyoto University of Education.
His research interests are human informatics
and intelligence science.

Maki Sakamoto acquired a Ph.D. from the University of Tokyo in 2000. Since 2015, she has
been a professor at the University of ElectroCommunications, Chofu, Japan.