2019 IEEE 1st Global Conference on Life Sciences and Technologies (LifeTech 2019)

Vector Auto-Regression Model of Temporal
Perceptual and Affective Responses Towards Food
Takumu Okada, Shogo Okamoto, Yoji Yamada

Tomoharu Ishikawa

Nagoya University, Japan

Utusnomiya University, Japan

Abstract—Although human perceptual and affective responses
triggered by the exposure to sensory experiences are dynamic,
their temporal evolution has not been intensively investigated.
In this study, we aimed to construct a mathematical model ofsuch dynamics. The perceptual and affective responses whileeating strawberries were measured using the temporal dominanceof sensations method. We modeled them with a vector autoregression (VAR) model based on Granger causality between
perceptual and affective responses.

I. I NTRODUCTION
Our affective experiences are changed by stimuli perceived
via our senses. For example, once we perceive the characteristic taste or smell of food, this sensory information is integrated
into affective sensations like satisfaction, or evaluations suchas like/dislike. In this way, the sensory information caused by
the exposure to stimuli changes our affective feelings in a dynamical process. However, methods to mathematically model
these dynamic affective responses have not been established.
Some researchers specified the semantic causal relationships
among adjectives used in sensory evaluation [1], [2], [3],[4], and identified the causalities from the sensory adjectives
expressing physical characteristics of the stimuli to the affective adjectives representing their affective characteristics. The
causalities from the sensory or affective adjectives to the evaluative adjectives expressing preference were also observed.
This multi-layered causal structure among senses, affects
and preferences implies that our affect and preference dynamically change, being triggered by the stimuli perceived via oursenses. In these experiments, however, subjects were allowed
to evaluate each adjective at any time in sensory evaluation,that is, the causalities between perceptual and affective feelings
were not investigated in their temporal aspect.
In this research, we focus on taste and affective feelings
and aim to establish mathematical models of perceptual andaffective responses experienced immediately after starting toeat. We measure the temporal change of taste and affective
feelings using the temporal dominance of sensations method,and demonstrate that the resulting multivariate time series canbe treated with a vector auto-regression (VAR) model.
II. M EASUREMENT OF PERCEPTUAL AND AFFECTIVERESPONSES TOWARDS FOODS

The temporal dominance of sensations (TDS) method [5] isa sensory evaluation technique that can be used to efficiently
measure multiple sensations caused by eating food. In the TDS

978-1-7281-0543-7/18/$31.00 ©2019 IEEEFig. 1. A graphical user interface used in the TDS method.

method, a graphical user interface such as the one shown in
Fig. 1 is used. On the computer screen, some buttons withadjectives such as “sweet” or “sour” are displayed. Firstly,
the panelists push the start button as soon as they put foodinto their mouth. While eating the food, they select and push
the button with the sensation which they dominantly feel at
the moment. They push a button every time the dominantsensation changes until the food varnishes in the mouth, andthen push the stop button. The changes in perceptual andaffective responses are recorded as the sequence of dominantsensations.
III. E XPERIMENT
We performed a TDS experiment using strawberries. The
panelists were 7 males and 1 female.
A. Choice of adjectives
We selected the adjectives to be used in the experimentthrough a preliminary experiment performed by 7 participants.
As a result, 14 words, suitably representing the taste oraffective characteristics of strawberries, were selected out of
the 137 candidates we had prepared. Seven of them are sensory
adjectives which express the characteristics of food perceivedvia senses; “Sour,” “Sweet,” “Juicy,” “Melty,” “Refreshing,”“Soft,” and “Watery.” The other 7 were affective/evaluative adjectives representing affective or preferential feelings; “Like,”“Elegant,” “Natural,” “Delicious,” “Happy/Satisfied,” “Fresh,”and “Flavorous.” In the TDS experiment, adjectives belonging
to one type (sensory or affective/evaluative) at a time weredisplayed on the screen.

43

0Refreshing
Melty

Juicy

Observed

0.5
Normalized time(a) Change of sensations for sensory adjectives.

100806040
200

WaterySoftLike
Delicious

Flavorous
NaturalFresh
Elegant

Change ofdominance rate

Sweet
Sour1

Happy/Satisfied

Change ofdominance rate

Dominance rate [%]Dominance rate [%]100806040200(a) Like0.1

R = 0.69

0-0.1
00.2

0.1

0.4
0.6
Normalized time

0.8(b) Delicious

R = 0.75

0.40.6
Normalized time

0.8

10-0.1010.5
Normalized time(b) Change of sensations for affective/evaluative adjectives.

Estimation

00.21Fig. 3. Results of the estimation of affective responses using the VAR models.
(a) Estimation of “Like” by (1). (b) Estimation of “Delicious” by (2).

Fig. 2. Temporal change of perceptual and affective responses while eatingfood. The horizontal axis represents normalized time, spanning from the intake
of a strawberry (0) to the end of perceptions (1). (a) Results for sensory
adjectives. (b) Results for affective/evaluative adjectives.

B. Task
One strawberry was used for each trial. In each trial, thepanelists first rinsed their mouth with water. Immediately after
they put a strawberry in their mouth, they started the taskof pushing buttons. Each panelist performed this task threetimes for each type of adjective alternately. The locations ofadjectives on the screen were randomized for each panelist.
C. Results
The graphs shown in Fig. 2 show the perceptual and affective responses obtained in the experiment. The vertical axis
shows the dominance rate, which is the ratio of total numberof trials to the number of times each sensation was selected.
For sensory adjectives (Fig. 2(a)), the dominant sensations
were “Juicy” and “Sweet” in the first half and “Sour” inthe second half. For affective/evaluative adjectives (Fig. 2(b)),“Delicious” was the most dominant sensation during a largepart of the trial. Among other adjectives, “Flavorous” wasdominant in the first half of the trial and “Like” and “Fresh”
in the second half. On the other hand, “Soft,” “Melty,” “Elegant,” “Happy/Satisfied,” and “Natural” were excluded fromthe analysis of taste and affective feelings since they wereinfrequently selected throughout the experiment.
IV. M ATHEMATICAL MODEL OF AFFECTIVE DYNAMICSA. VAR model
We used vector auto-regression (VAR), often applied ineconomics and neurosciences to describe the evolution ofmultivariate time series, to model the perceptual and affective
responses. In a VAR model for n variables with a modelorder q, one of the variables at time t is expressed as thelinear function of the past q lags of the n variables, including

itself. Since the VAR model is applied to discretized data, thecontinuous data obtained by the TDS method were discretized
into 30 equally spaced time points (sampling period ∆t ≃ 1s).
In order to select the explanatory variables in the VAR
model, we used Granger causality, which can statisticallytest causal relationships between time series. As explanatory
variables in a VAR model to estimate a certain sensation, weelected only those sensations from which the causality to thetarget sensation was significantly supported (p < 0.10). We
analyzed the time series shown in Fig. 2 after removing theirtrend by differencing, since the Granger causality test assumes
stationarity of the data. The model order of the VAR model
was selected based on information criterion to each estimatedvariable.
B. Result
We report the results for two affective/evaluative adjectives,“Like” and “Delicious”. The results of the Granger causality
test suggested that “Like” was negatively affected by “Watery”(sensory) and positively affected by “Fresh” (affective) and“Flavorous” (affective). The results also suggested that “Delicious” was positively affected by “Sweet” (sensory). Some
previous researches [1], [2] also suggested that preferentialadjectives such as “Like” were affected by not only sensory adjectives but also affective ones. Figure 3 shows thechanges in “Like” and “Delicious” estimated by the VAR
models including only the variables for which causality toeach sensation was supported. The correlation coefficients
between estimated and observed values were R = 0.69 and
R = 0.75 respectively. The VAR models (q = 1) for “Like”
and “Delicious” are the following:

44ylike,t = 0.011 − 0.99ywatery,t−∆t + 0.15yfresh,t−∆th+ 0.25yflav,t−∆t + 0.17ylike,t−∆t ,(1)ydel,t = 0.0026 + 0.43ysweet,t−∆t + 0.41ydel,t−∆t .

(2)

These results show that the change in the feeling of “Delicious” was mainly caused by the sense of sweetness. Althoughmodel (1) had two more explanatory variables than model(2), the correlation coefficient between observed and estimatedvalues was larger for (2), implying that “Like” is a sensation
more complex and difficult to predict compared to “Delicious.”
V. C ONCLUSION
We measured perceptual and affective responses while eating food using the TDS method and mathematically estimatedthe affective responses from the perceptual ones. Simple andeffective models of affective responses were established byselecting the explanatory variables included in a VAR model
considering Granger causality in the choice of variables.
Mathematical models of affective responses may enable us
to design the changes in the affective feelings of consumers.
ACKNOWLEDGMENT
This study was in part supported by MEXT Kakenhi(17K20002).
R EFERENCES[1] H. Nagano, S. Okamoto, and Y. Yamada, “Modeling semantically multilayered affective and psychophysical responses toward tactile textures,”IEEE Transactions on Haptics, 2018.
[2] S. Okamoto, H. Nagano, K. Kidoma, and Y. Yamada, “Specification ofindividuality in causal relationships among texture-related attributes, emotions, and preferences,” International Journal of Affective Engineering,
vol. 15, 2016, no. 1, pp. 11–19.
[3] K. Kidoma, S. Okamoto, H. Nagano, and Y. Yamada, “Graphicalmodeling method of texture-related affective and perceptual responses,”
International Journal of Affective Engineering, vol. 16, 2017, no. 1, pp.
27–36.
[4] S. Okamoto, H. Kojima, A. Yamagishi, K. Kato, and A. Tamada,“Layered-modeling of affective and sensory experiences using structural
equation modeling: Touch experiences of plastic surfaces as an example,”
IEEE Transactions on Affective Computing, 2018.
[5] N. Pineau, P. Schlich, S. Cordelle, C. Mathonnière, S. Issanchou, A. Imbert, M. Rogeaux, P. Etiévant, and E. Köster, “Temporal dominance ofsensations: Construction of the tds curves and comparison with time–intensity,” Food Quality and Preference, vol. 20, 2009, no. 6, pp. 450–455.

45