2021 IEEE/SICE International Symposium on System Integrations (SII)
Iwaki, Fukushima, Japan. January 11 - 14, 2021

Hierarchical Modeling of Individual Tactile, Emotional, andPreferential Responses to Leathers using Bayesian Structural
Equation Modeling*
Shin Inami*1, Shogo Okamoto*2, Kunitoshi Ito*1, and Ryuji Ozaki*1
multivariate analysis to the characteristics of the material
samples obtained from commercially-available measuring
instruments and the average of the corresponding results frompanelists’ questionnaires. Despite the many previous studies
conducted in this field, unsolved problems remain. There are
two problems with earlier approaches: 1) the reliability of thequestionnaire is low, and 2) the causal relationship is notrevealed by the multivariate model.

Abstract— In order to increase the emotional value of
products when being touched, it is important to be able toexplain the relationship between physical phenomena given to
end users and their perceptual and emotional responses andpreferences. Although general methods in current modeling ismultivariate analysis to the physical characteristics of material
samples obtained from commercial measuring instruments, they
do not address two kinds of variances entangled. First, thedeviation is caused by individual differences in the finger’s
mechanical properties and hand motions to explore material
surfaces. Second, the repetitive error of the answers is caused by
the difficulty of the questionnaire survey. Therefore, wemeasured the physical (mechanical and thermal) phenomena
between the leather material and the panelists actually touching
the sample and added them to the explanatory variable. Then,
using latent factors of structural equation modeling (SEM), therepetitive error of the questionnaire was attempted to be taken
into account while modeling the hierarchical structure of
low-order perceptions such as “soft” and “smooth” andhigher-order emotional responses and preferences such as“luxury” and “comfortable.” In the conventional SEM, in casethe large estimation error is set on an observed variable, thecalculation becomes unstable due to the local solution. In thisstudy, we applied a method to stabilize calculation whileconsidering a large error using the Bayesian SEM. Thisapproach produced a semantically reasonable model that would
help design leather products.

Here is an example expanding on the issue of thereliability of the questionnaire. In the past, we conducted amultivariate analysis on 15 samples of genuine and synthetic
leather using the mechanical properties measured withcommercial measurement instruments as explanatory
variables (e.g. KES series, Kato Tech Co., Ltd.), and thequestionnaire results from panelists as objective variables.
The regression to the average value for the questionnaire had ahigh coefficient of determination of 0.878, but the regression
to all responses for the questionnaire had a low coefficient ofdetermination of 0.151. The problem with this approach is thatit does not consider the individual differences in the responses
to the questionnaire or in the contact between the panelists’fingers and the materials.
Many previous studies have used multiple regression
models and have not aimed to identify causal relationships
between variables. For example, even if a positive correlation
between bending rigidity in a woven fabric and an impressionof luxury is found, their causal relationship is not shown. In
contrast, hierarchical modeling is used [1—5] to further
explore causal relationships. In hierarchical modeling, forexample, the following relationships are specified. A woven
fabric with a lower bending rigidity feels softer. Softer fabrics
feel more comfortable. Finally, a comfortable woven fabric is
perceived to be luxurious. Such hierarchical relationships leadto an understanding of semantic causation. Nonetheless,whether they really show the cognitive relationship ofperception, emotion, and preference of physical properties
will be an object of debate for future studies. The present
paper adopts a hierarchical modeling approach.

I. INTRODUCTION
The automobile industry is undergoing a substantialchange described as a once in every 100 years majortransformation. The inevitability of this change is represented
by CASE, which stands for connected (C), autonomous (A),shared (S), and electric (E). At Toyota Boshoku, humanity(H) has been added to CASE to define CHASE as anopportunity to connect products and services to humanemotion and empathy and enhance the value they create,
which is represented by the Quality of Time and Space slogan.
As one of the methods to provide that value, it is necessary to
be able to evaluate the perceptual/emotional responses andpreferences of end users, such as luxury and comfort, using
measurable indicators. The present study relates the physicalcharacteristics of materials and the subjective responses about
perception, emotion and preference they instill in users,
focusing particularly on tactile sensation over the other four
senses. This has typically been approached by applying

In light of the above problems, we developed a modelingmethod that incorporates the following two points. To our best
knowledge, there is no modeling method solving these
problems. First, most previous studies used commercial
measurement instruments, but in order to explicitly consider
the difference between panelists, we decided to measure themechanical and thermal contact phenomena when they touch
the material (Sec. II-A). Next, we investigated the consistencyof the responses of the panelists themselves, and used the highconsistency responses of the panelists. However, it was*1: Interior & Exterior Evaluation & Engineering div, Toyota-Boshoku
Corporation. Email: shin.inami@toyota-boshoku.com*2: Dept. Mechanical Systems Eng. Graduate School of Engineering,
Nagoya Univ.
978-1-7281-7657-4/21/$31.00 ©2021 IEEE745

impossible to reduce the error as much as expected. Bayesian
SEM was adopted as a hierarchical modeling method to takethe repetitive error of these questionnaires into account (Sec.
II-B).

Table 1. Classification of thirty leather samples
M aterial
FeatureGenuineSynthetic

II. DATA AND METHOD
In the previous section, we noted that individual
differences in finger conditions and hand motions have notbeen incorporated into the model. Further, the repetitive error
in questionnaire has not been considered in the model. In thissection, we describe solutions to address these issues.

GrainedShrink
Fine
8
7
4
4

Suede
4
3

Table 2. Questions on perception, emotion and preference
Perception-Soft +Hard-Warm +Cold-Resilient +Flexible-Smooth +Rough-Bumpy +Flat-Scratchy +NotScratchy-With +W/O Tension-Moist +Dry

A. Individual differences in finger conditions and touch
behaviors
As mentioned above, most previous studies usedcommercial measurement instruments to obtain mechanicalproperties of samples, but we considered that the differences
between hands and touch methods can be incorporated in themodel as an explanatory variable by measuring the physicalphenomena when a panelist actually touches the sample. For
each panelist, ten random leather samples were picked at
random from a selection of thirty leather samples (Table 1).
Then, physical measurements and the questionnaire wereconducted for blind-folded panelists. The physicalphenomena measured, chosen by referring to tactile
dimensions for textures [6], were the load-displacement
property when the sample is pushed with the tip of the index
finger, the temperature-time property of the finger and sample
interface to acquire heat-transfer characteristics, as well as thevibration-frequency property when the fingertip slides overthe sample, and friction-speed property (Stribeck
characteristics) when the sample is stroked with the tip of theindex finger. For measurement, we prepared a three-axis loadmeter, a two-dimensional laser displacement meter thatmeasured the position of the finger, a small thermistor and an
acceleration detector. The feature quantities were collectedfrom the measured waveform data. However, the number offeature dimensions rose as high as 140, and then exploratory
factor analysis was performed on each of the four measured
physical stimulus waveforms, to reduce the number ofdimensions to 21. By contract, some details cannot be
disclosed, regarding the measurement equipment and feature
dimensions.

Emotion/Preference-Gorgeous +Concise-Elegant +Inelegant-Solemn +Relaxed-Urban +Country-Cute +NotCute-Friendly +Hostility-Obedient +Stubborn-Young +Old
-Luxury +Cheap-Conmfort +Uncomfort

Fig. 1 Histgram of coefficients of determination as
consistency of panelists’ answers. The higher, the moreconsistent.
concerning the perception of physical properties (“soft-hard,”“warm-cold,” etc.) and ten questions addressing emotion andpreference (“luxury-cheap,” “cute-not cute,” etc.), using the
7-point semantic differential method. During thequestionnaire, we randomly presented specific samples twice
without notifying the panelists, and received pair of responses
to the same sample. Since the distribution of the coefficient ofdetermination of these responses was divided in two peaks at
0.30 (Fig. 1), it was decided to use only the data from panelists
with coefficient of determination above 0.30.
In addition, a structural equation model (SEM) was
adopted as the hierarchical modeling method (using the R
lavaan package [7]). The reason for that choice is that thecausal relationship can be analyzed hierarchically using a path
diagram, and modeling can be performed while taking
repetitive error into account by setting latent factors. As
mentioned in Section I, it is impossible to completely remove
the repetitive error by selecting the data of the panelists withconsistent answers. As shown in Fig. 1, even amongprofessional panelists who regularly handle the material, it israther rare for the panelists to ensure consistency in the task ofscoring the tactile physical phenomena. For example, only 3%of all panelists have a coefficient of determination above 0.80.

B. Repetitive error due to difficulty of the questionnaire
In order to reduce the effects of the repetitive error of thequestionnaire, we considered excluding the data of panelists
who cannot determine tactile sensation with consistency fromthe analysis. First of all, we tried to improve the accuracy ofthe questionnaire by asking panelists working in the leather
industry, who have many opportunities to come in contact
with leather material on a daily basis. This might bias the dataand eliminate the generality of the model. However, in thisstudy, the range of panelists was narrowed down to lessen therepetitive errors in the questionnaire, because the errors, i.e,
random variations, of the answers from in-house panelists waslarge in the pre-survey. In order to ensure the generality of themodel, we need to collect data from various panelists in the
future. The questionnaire consisted of a total of eighteen
questions (Table 2) in a random order, with eight questions

Furthermore, in the SEM calculation built on the strictassumption that most of the variance in the observations isrepetitive error, the search for the solution using the maximum
746

Fig. 2 Schematic diagram of SEM.
likelihood method tends to make regression error of specificnode to converge to zero, path coefficients diverge and it
outputs improper solution. Therefore, in this study, theparameters were estimated using a Bayesian SEM that canhandle Markov chain Monte Carlo methods (MCMC), using
the R blavaan package [8], [9]. Bayesian SEM differs from the
maximum likelihood method used in ordinary SEM bygenerating proposed parameter through a Markov chain andadopting or rejecting them on the basis of likelihood.
Repeating this operation and performing the Monte Carlo
method, produces a histogram of adopted values (whichconstitutes the posterior probability distribution). Although
the computational cost is high, this parameter estimation
method does not get caught in the local minimum. In this study,
we realized a robust hierarchical causal model by including
latent factors in Bayesian SEM. This also makes it possible toaccount for the magnitude of the repetitive error obtained
from the actual data in the analysis. As mentioned above, mostprevious studies did not incorporate mere repetitive error due
to difficulty of the questionnaire. Instead, they used averageanswers from multiple trials or they did not retest the same
sample for the same panelist.
C. Construction of 3-layered model
The SEM of this study has a three-layer structure in which
the feature quantities of the physical phenomena are placed in
the lower layer, the latent variables of the questionnaire results
regarding low-order perception are placed in the middle layer,
and the latent variables of the questionnaire results regarding
emotion and preference are placed in the upper layer (Fig. 2).
747

In order to determine the placement of latent factors, an
exploratory factor analysis was performed on the eightmeasured variables about perception in the middle layer, andbased on a scree plot (a graph the eigenvalues of thecovariance matrix plotted in descending order), we
determined the number of latent factors to be three. The threefactor scores and loadings were then calculated as in Table 3.
After that, as a preliminary analysis for designing the path
diagram, the above three factor scores were assigned to the
middle layer, and a normal SEM analysis was performed
without latent factors. The values of several fit indices werevery good, and the accuracy of the model configuration was
confirmed. Consequently, in order to incorporate therepetitive error of the questionnaire, the latent factors wereinserted in place of the middle lower-order perceptual layer
and the upper sensitivity and preference layer in thethree-layer model built on this configuration. In the lower
perceptual layer, the three latent factors correspond to theresults of the exploratory factor analysis (Table 3) above.
In the upper emotion and preference group, the scores forthe two factors of “gorgeous, elegant, solemn” and “cute,friendly, obedient” in the emotional questionnaire items were
derived in advance and added to the model. The latent factorsof “luxury” and “comfort” were defined as well using the same
method (Fig. 3). We defined 0.200 (this value should be
optimized in the future) as a provisional value smaller than the
average error variance of 0.580 for the data in Fig. 1 as theerror variance from these latent factors to the observed
variables. Then, the maximum likelihood method was
attempted to calculate this model. However, with a normal

SEM analysis, despite smaller assumed error variances thanthe estimated values by actual measurement, the regression
error converged to zero and the coefficients of paths to node
FZ2 “luxury” in the emotional and preference layer resulted
in improper solutions. In order to perform similar parameter
estimation with Bayesian SEM, this time from the blavaan
package in conjunction with the JAGS software for the Gibbs
sampling, that constitutes one of MCMC methods, burn-in
was applied 1000 times, model adapting 1000 times, anditeration 10000 times to estimate each parameter.

Table 3. Loadings of three factors for the perceptual layer.
Left column lists the eight types of perceptual items inquestionnaire.
-Rough/Bumpy-Soft/Warm+Smooth/Flat(FY1) +Hard/Cold(FY2)-Soft +Hard
0.05
0.83
-Warm +Cold
-0.04
0.79
-Resilient +Flexible
0.16
0.22
-Smooth +Rough
-0.84
-0.03
-Bumpy +Flat
0.75
-0.31
-Scratchy +NotScratchy
0.89
0.11
-With +W/O Tension
-0.34
-0.40
-Moist +Dry
-0.39
0.52

III. RESULTS
The parameter estimation converged with a model
configuration that could not be solved by the maximum
likelihood method as in Fig. 4. The maximum value of thepotential scale reduction factor (PSRF) of the model was
1.055, which indicates the degree of convergence of thecalculation for the difference between the three Markov
chains. Generally, parameters converge when equivalent to1.050. Thus, Bayesian SEM can model the complex
relationship of tactile sensations that cannot be estimated byordinary SEM while taking repetitive errors into account. The
result of the analysis showed that the regression residual valueof the luxury latent factor was 0.137, and the coefficient ofdetermination was equivalent to 0.863. This model exhibits
the accuracy close to the average value of the multivariate
analysis in Section I.
We investigated each path coefficient to interpret thederived model. For example, the regression path coefficientsfor the latent factor of FY1 taken from the relationshipbetween the lower physical layer and the perceptual layer, areshown in Table 4. For example, a positive coefficient fromMR1 (normal force while sliding) and MR14 (normal force
while pushing) to FY1 suggests that panelists applied larger
normal forces for bumpier surfaces. In other words, by
touching flat surface with smaller normal force, they might
have avoided unpleasantness resulting from stick-slip
phenomenon (alternatively repeating static and kinetic
friction) and risk of damaging or wrinkling the material intheir experience. Negative effects from MR2 (shearing force
while sliding) and MR15 (coefficient of friction while sliding)on FY1 indicate that bumpier surfaces produced smaller
friction. A positive effect MR5 (magnitude of vibration above
50 Hz) to FY1 indicates the vibration becomes large when
getting over the roughness, and A positive effect MR9(Kurtosis of linear regression residuals of vibration spectrum)
to FY1 indicates the spectrum becomes peaky due to theperiodicity of the unevenness. These coefficients arereasonably interpreted. We also checked path coefficientsfrom the perceptual layer to emotional and preference layer(Table 5). Both “luxury” and “comfort” factors received thelarge effects from the factor of “flat, not-sticky, moist, soft,
and warm,” which seems reasonable for leathers. Nonetheless,its semantic validity should be thoroughly investigated in the
future.

-Resilient+Flexible(FY3)
0.24
0.02
0.99
-0.02
0.01
0.11
0.25
-0.06

Fig. 3 Preliminary SEM path diagram

IV. DISCUSSION
It is noteworthy that the “luxury” and “comfort” factors
are very similar as in Table 3. In the previous survey, when theFig. 4 Path diagram of Bayesian SEM
748

same questionnaire was given to in-house Japanese panelists
and to European local base panelists, the Japanese panelists
preferred only the comfort factor, while European panelists
tended to prefer both the luxury and comfort factors. It isinteresting that Japanese leather professional panelists
exhibited the same evaluation tendency as the European
panelists. This suggests that narrowing down the data basedon user group or on location and age in future research, will
make it possible to create an evaluation index close to market
needs. In order to realize the evaluation index, it is necessary
to collect data from various panelists in the future. Onceevaluation indexes tailored to a wide variety of users are
created from the collected data, the next step will be to classify
the tendencies in end users’ subjective responses andpreferences, and build technology to predict them. This is
likely to contribute to further improving the value associatedwith emotional and preferential responses.

Table 4. Regression model for FY1+Rough/Bumpy/Scratchy-Smooth/Flat/NotScratchy (FY1)
Normal force while sliding (MR1)
Shearing force while sliding (MR2)
Magnitude of vibration above 50Hz (MR5)
Kurtosis of linear regression residualsof vibration spectrum (MR9)Normal force while pushing (MR14)
Coefficient of friction while pushing (MR15)

Perception

[7]
[8]
[9]

ACKNOWLEDGMENT
We would like to thank all the people in the leather industry
who participated in collecting data despite their busy
schedules for their generous cooperation. We are fullycommitted to continuing to do our utmost to provide end users
with the value defined by Quality of Time and Space.
REFERENCES

[3]

[4]
[5]

[6]

Emotion/Preference+Rough/Bumpy/Scratchy-Smooth/Flat/NotScratchy (FY1)+Hard/Cold/Dry-Soft/Warm/Moist (FY2)+Flexible-Resilient (FY3)

Incorporating not only the physical property data collectedby commercial measurement instruments but also the contact
phenomena produced by the individual panelists as exogenous
variables allowed us to model the causal relationship between
perceptual responses, individual hand states, and touch
behaviors. Bayesian SEM enabled hierarchical modeling that
accounts for large repetitive errors that cannot be estimatedusing conventional SEM. As a result of incorporating theabove two points, it has become possible to separate andmodel individual differences in physical phenomena andrepetitive errors in answers to questionnaires.

[2]

0.280
-0.261
0.322
0.238
0.283
-0.252

Table 5. Regression model for “luxury and comfort”

V. CONCLUSION[1]Path coefficient

M. Murakami, T. Fujimoto, M. Mori, and S. Tandon, “Effect of various
finishing treatments on handle and appearance of wool tweed fabrics,”
International Journal of Affective Engineering, vol. 16, no. 2, pp.
194—155, 2017.
I. H. M. Hashim, S. Kumamoto, K. Takemura, T. Maeno, S. Okuda,
and Y. Mori, “Tactile evaluation feedback system for multi-layered
structure inspired by human tactile perception mechanism,” Sensors,
vol. 17, 2601, 2017.
S. Okamoto, H. Kojima, A. Yamagishi, K. Kato, and A. Tamada,“Layered-modeling of affective and sensory experiences using
structural equation modeling: Touch experiences of plastic surfaces as
an example,” IEEE Transactions on Affective Computing, 2019.
S. Kawabata and M. Niwa, “Fabric performance in clothing andclothing manufacture,” Journal of Textile Institute, vol. 80, no. 1 pp.
19—50, 1989.
X, Chen, C. J. Barnes, T. H. C. Childs, B. Henson, and F. Shao,“Materials’ tactile testing and characterization for consumer products’
affective packaging design,” Materials and Design, vol. 30, pp.
4299—4310, 2009.
S. Okamoto, H. Nagano, and Y. Yamada, “Psychophysical dimensionsof tactile perception of textures,” IEEE Transactions on Haptics, vol.6,
no.1, pp.81-93, 2013.
749

Path coefficient+Cheap-Luxury (FZ1)+Uncomfortable-Comfortable (FZ2)

0.525

0.404

0.769

0.788

-

0.090

Y. Rosseel, “lavaan: An R package for structural equation modeling,”
Journal of Statistical Software, vol. 48, p.2, 2012.
E. C. Merkle, Y. Rosseel. “blavaan: Bayesian structural equation
models via parameter expansion” Journal of Statistical Software, vol.
85 p.4, 2018.
B. Muthen and T. Asparouhov, “Bayesian structural equation
modeling: A more flexible representation of substantive theory,”
Psychological Methods, vol.17 No3, pp. 313–335,2012.