Layered Modeling of Sensory and Affective
Responses: Modiﬁcation by Considering
Unique Factors of Affective Responses
Nanako Shimaoka, Shogo Okamoto(&), Kyoichi Tachi,
Yasuhiro Akiyama, and Yoji Yamada
Nagoya University, Nagoya, Aichi, Japan
shogo.okamoto@mae.nagoya-u.ac.jp

Abstract. Layered modeling is used to understand the semantic structure ofsensory and affective responses acquired through the sensory evaluation ofproducts. In this paper, we propose a method for establishing layered models,
which considers unique factors, that is, unpredictable components independentof other variables. When creating a layered model, the subjective rate to eachverbal criterion of sensory evaluation is treated as a variable, and the variables
with the lowest estimation accuracy are remodeled. Conventionally, the determination coefﬁcient of each variable has been used as a criterion for estimation
accuracy. However, this value is influenced by unique factors, including random
repetition errors. This influence should be removed because the estimation ofunique factors is not improved by modifying the model structure. We consider
the influence of unique factors of a certain variable by using the determination
coefﬁcient of linear regression by using the other variables. We applied themodiﬁed modeling method to the sensory evaluation of miso, or fermented
soybean paste.
Keywords: Structural equation modeling  Sensory evaluation  Miso

1 Introduction
In understanding and designing the affective responses experienced from products, it isuseful to mathematically model the relationship between the sensory and affectiveresponses. Layered modeling is one of the methods used to establish such models [1–6]. Figure 1 illustrates an example of a layered model for tasting experiences. Eachnode corresponds to the score assigned to a verbal criterion, which is typically anadjective. For example, the score for mild is expressed as follows:
ymild ¼ 0:31xbitter þ 0:25xumami þ e

ð1Þwhere x is the score for sensory word, y is the score for affective and evaluative words,
and e is a random error. This formula indicates that food with little bitterness and moreumami is considered milder. The layered model expresses the semantic structure ofsensory and affective responses in an easy-to-understand manner.
© Springer Nature Singapore Pte Ltd. 2020
H. Shoji et al. (Eds.): KEER 2020, AISC 1256, pp. 168–173, 2020.
https://doi.org/10.1007/978-981-15-7801-4_17

Layered Modeling of Sensory and Affective Responses

flavorsome (.28)

delicious (.30)
.20

.18

.23

mild (.51)-.31bitter169

Affective andevaluativevariables

rich (.40)
.25

.18

umami.23
sweetLow levelsensory
variables

Fig. 1. Layered model of sensory and affective responses to foods.

Figure 1 shows an example of a layered model of sensory and affective responses
to foods. The value beside the edge is the strength of the relationship between twovariables. The value following the adjective is the R2 value, which is the estimation
accuracy of the variables for the affective and evaluative words. The variables in thebottom layers are correlated to each other.
There are several methods for establishing a layered model. The method in [1]automatically determines the structure of the model, including the number of layers,and validates the model by structural equation modeling, whereas layered models aredesigned or hypothesized by experienced assessors in other methods. However, there isa concern with the method in [1]. In the process of creating a layered model, variables
with the lowest estimation accuracy are remodeled. Conventionally, the determination
coefﬁcient of each variable has been used as a criterion for estimation accuracy.
However, this value is influenced by unique factors (i.e., unpredictable components thatare independent of other variables). Unique factors include random repetition errors.
This component should be considered when modeling (see Sect. 2.2), which themethod used in [1] did not. In case that the variables include large unique factors, theprevious approach may establish incorrect models. In this paper, we propose a method
for establishing layered models that consider unique factors, and we apply the method
to the sensory evaluation of miso, or fermented soybean paste.

2 Method: Layered Modeling
2.1

Conventional Modeling Method

First, structural equation modeling (SEM) is applied to a two-layered model in which
the low-level sensory items are explanatory variables, and the upper-level affective andevaluative items are objective variables, as shown on the left in Fig. 2. The explanatory
variable with the lowest estimation accuracy, R2 , is placed in a higher layer to beexplained by other affective and evaluative variables, as shown on the right in Fig. 2.
By repeating this operation until the statistical criteria of SEM are satisﬁed, a multilayer model can be obtained.

170

N. Shimaoka et al.
rich (.63).20
mild (.51)-.31bittermild (.51)

rich (.40)
.25

.23

.18

-.31
bitter

sweet

umami

.25.18

umami

.23

sweet

Fig. 2. Stepwise development of the layered model [1]. The variable with the lowest value in theparentheses (i.e., rich) is relocated into a higher layer.

2.2

Modiﬁcation by Considering Unique Factors

As mentioned in the Introduction, the method using R2 , as shown in Sect. 2.1, does notconsider the presence of unique factors that are independent of other components.
Unique factors are unpredictable by modifying the model structure, so when creating amodel, their effects should be removed. Therefore, the objective variable is predictedby linear regression of all other variables as shown in (2), and the coefﬁcient of^ 2 , is introduced. This value is the maximum estimation accuracy ofdetermination, R
linear predictive models in principle.
^yi ¼

Xaji yj þ

Xbji xj

ð2ÞPnð^yik  yi Þ2
2^
Ri ¼ Pk¼1
nyi Þ 2k¼1 ðyik  

ð3Þ

i6¼j

where x is the low-level sensory variable, y is the upper-level affective and evaluative
variable, ^y is the estimated y value, y is the mean of y, aji and bji are the coefﬁcients thatshow the strength of the connection between variable i and variable j, and n is thenumber of samples.
^ 2 is the one with poor estimation
The variable with the smallest ratio of R2 to Raccuracy. Therefore, in the model considering the unique factors, a model is established^ 2 instead of R2 .
using R2 =R

3 Example Using the Sensory Evaluation of Miso (FermentedSoybean Paste)
3.1

The Sensory Evaluation of Miso

Participants tasted twelve kinds of miso and scored eleven low-level sensory items, andnine affective and evaluative items in seven levels. These items were selected among
approximately 200 potential words used for expressing foods by voting tasks involving
several assessors. The low-level sensory items are represented by uni-sensory organs

Layered Modeling of Sensory and Affective Responses

171

and they represent gustatory, osmatic, and textural aspects, including salty, sour, bitter,melty, sticky, rough, fermented, umami, moist, astringent, and soy sauce-flavored. The
affective and evaluative items are composed of the integration of multiple sensory
channels, feelings, and overall signiﬁcance or importance, which includes delicious,simple, mild, sharp, rich, flavorsome, mellow, reﬁned, and light. The meanings or
deﬁnitions of these words were well explained to the participants before the rating task.
They rinsed their mouth with tea before each evaluation so that the evaluation would
not be influenced by the miso that had been eaten before.
3.2

Participants

The participants were nine people from Nagoya University. None of them studied orworked in the food industry.
3.3

Established Layered Models

The obtained evaluations were standardized for each individual and item, and modeledusing the method described above. Figure 3 shows the layered model created by theconventional method, and Fig. 4 shows the layered model considering the unique^ 2 . We present only statisticallyfactors. The values next to the word are R2 and R2 =R
signiﬁcant links. The bottom variables are correlated.

mellow (.16, .73)

refined (.15, .54)

rich (.13, .44)

.39

simple (.32, .72)-.50

delicious (.26, .51)
.51
mild (.13, .25)

.49

-.20

light (.16, .35)
.40

-.16
-.15
-.34.18sticky

sensory-.36-.31.29).18

flavorsome (.26, 75)
.24

e.g.
affective (-.19.21.19

sharp (.27, .77)

fermented

bitterFig. 3. Layered model established by the conventional method GFI = 0.95, CFI = 1.00, andv2 = 43.7 df ¼ 45 (p < 0.53).

172

N. Shimaoka et al.

flavorsome (.28, 83)
.17
.27

.20

-.26

mellow (.13, .57)
.26-.25

refined (.15, .54)

simple (.31, .69)
.45.39delicious (.26, .61)-.20-.50
mild (.13, .25)-.34bitter-.22

sharp (.26, .75)
.51

.18

light (.26, .55)

.37
-.30
rich (.03, .11)

.18

.18

stickyfermented

.14

umami

Fig. 4. Layered model established by the new method GFI = 0.95, CFI = 1.00, and v2 = 50.3
df ¼ 53 (p < 0.58).

3.4

Elementary Screening for the Semantic Validity of Models

As shown in Figs. 3 and 4, in both types of models, the stickier and less bitter miso
were judged to be mild. Mild positively affected delicious, and delicious miso wasjudged as reﬁned and flavorsome. Mild miso was judged as light, and light positively
affected simplicity. A simple miso was not judged as flavorsome or mellow. These
results are semantically reasonable.
Some words were placed on different layers between the two models. For example,
simple was placed on a lower layer than rich (the taste which is composed of a greaternumber of gustatory factors) in the model established by the conventional method,whereas rich is placed in a lower layer than simple in the model established by the newmethod. As a result, the end of the edge from fermented is simple in Fig. 3, whereas itis rich in Fig. 4. Although their semantic relationships were similar, fermented directlyaffected simple negatively in Fig. 3, but indirectly in Fig. 4. Similarly, fermented
indirectly affected rich positively in Fig. 3, but directly in Fig. 4. However, simple andrich had a negative relationship.
The two types of models largely agree with each other, except for umami. In themodel established by the conventional method, as shown in Fig. 3, umami was not
signiﬁcantly linked with the other variables; however, umami positively influenced
light in the model established by the new method, as shown in Fig. 4. Nonetheless, thisinfluence was not potent in terms of the coefﬁcient value (.14) between umami andlight. To systematically compare the semantic validity of these two types of models, amethod used in [7] can be used, but remains to be studied in the future. In [7], several
assessors compared a few layered models regarding their semantic validity.

Layered Modeling of Sensory and Affective Responses

173

4 Conclusion
We modiﬁed a method of layered modeling for sensory and affective responses, which
was originally developed in [1]. The modiﬁed method establishes a layered model
based on net R2, excluding the effects of unique factors that cannot be predicted by thecombination of variables in the model. We used food samples as an example, but thismethod is expected to be applied to other types of stimuli.
Acknowledgment. This study was supported in part by MEXT KAKENHI (17H04697).

References
1. Okamoto, S., Kojima, H., Yamagishi, A., Kato, K., Tamada, A.: Layered-modeling ofaffective and sensory experiences using structural equation modeling: touch experiences ofplastic surfaces as an example. IEEE Trans. Affective Comput. (2018). https://doi.org/10.
1109/TAFFC.2018.2879944
2. Nagano, H., Okamoto, S., Yamada, Y.: Modeling semantically multilayered affective andpsychophysical responses toward tactile textures. IEEE Trans. Haptics 11(4), 568–578 (2018)
3. Chen, X., Barnes, C.J., Childs, T.H.C., Henson, B., Shao, F.: Materials’ tactile testing andcharacterisation for consumer products’ affective packaging design. Mater. Des. 30(10),4299–4310 (2009)4. Hashim, I.H.M., Kumamoto, S., Takemura, K., Maeno, T., Okuda, S., Mori, Y.: Tactile
evaluation feedback system for multi-layered structure inspired by human tactile perception
mechanism. Sensors 17(11), 2601 (2017)
5. Okamoto, S., Nagano, H., Kidoma, K., Yamada, Y.: Speciﬁcation of individuality in causal
relationships among texture-related attributes, emotions, and preferences. Int. J. Affective
Eng. 15(1), 11–19 (2016)
6. Kidoma, K., Okamoto, S., Nagano, H., Yamada, Y.: Graphical modeling method of texturerelated affective and perceptual responses. Int. J. Affective Eng. 16(1), 27–36 (2017)
7. Okada, T., Okamoto, S., Yamada, Y.: Affective dynamics: causality modeling of temporally
evolving perceptual and affective responses. IEEE Trans. Affective Comput. (2019). https://doi.org/10.1109/TAFFC.2019.2942931