This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TAFFC.2019.2942931, IEEE
Transactions on Affective Computing
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 20191

Affective Dynamics: Causality Modeling of
Temporally Evolving Perceptual and
Affective Responses
Takumu Okada, Shogo Okamoto, Member, IEEE, and Yoji Yamada, Member, IEEE
Abstract—Human perceptual and affective responses change dynamically when stimuli are experienced. In this study, we developed amethod for modeling the causal structures of affective dynamics using time-series data. Using the temporal dominance of sensations
method, perceptual and affective data were collected from individuals eating strawberries, and the resulting time-series data weremathematically represented using a vector auto-regression model. Multihierarchical and multidimensional causality structures that
explain the temporal evolution of perceptual and affective responses were then established based on Granger causality and theinformation criterion. The established model suggests how affective and preferential responses are triggered following exposure tostimuli. We also assessed the quantitative and semantic validity of the model.
Index Terms—Causality modeling, temporal dominance of sensations, VAR model, Granger causality.

F

1

I NTRODUCTION

Human affective response, or emotional response to asituation or stimulus, has gained increasing attention as atool for adding value to industrial products or services.
To translate abstract consumers’ affective responses into
physical quantities that can be accessed and controlled bydesigners, the relationships between consumers’ perceptualand affective responses toward products should be clarified. To date, a number of methods for constructing causal
models to visually express the causalities between humanperceptual and affective responses have been proposed [1],[2], [3], [4], [5], [6], [7]. Under the perceptual and affective
causal model, adjectives1 used in sensory evaluation as evaluation criteria are given as causalities from the lowest to theuppermost layers, as shown in Fig. 1. In this model, sensory
adjectives representing the physical characteristics of stimuli
are located at the bottom layer, while adjectives expressing
affective responses toward stimuli and preferential words
representing individual preference or evaluative words arelocated at the middle layer and top layers, respectively.
Using this structure, the causalities among the adjectives
can be investigated based on the evaluation values assigned
to different adjectives in various sensory evaluation experiments such as semantic differential methods and sensory
profiling. So far, for computing the structures, correlation
coefficient-based methods using computed evaluation values [1], covariance selection [3], and structural equation
modeling [4] have been suggested. In a different approach,panelists have been asked to directly identify the semantic•

T. Okada, S. Okamoto and Y. Yamada are with the Department of
Mechanical Systems Engineering, Nagoya University. This research wasapproved by the internal review board of the School of Engineering,
Nagoya University (#15–12).

Manuscript received April xx, 2019; revised August xx, 2019.
1. The descriptive norms are not limited to adjectives, but adjectives
are mostly used.

: Causal relationshipPreferential layer

Like/
DislikeGood/
Bad

Affective (emotional) layer

Interesting/
Uninteresting

Comfortable/
Uncomfortable

Psychophysical (sensory) layer

Rough/
Smooth

Hard/
Soft

䞉䞉䞉

䞉䞉䞉

䞉䞉䞉

Fig. 1. Example of causal adjective structure established from sensory
evaluation. Adapted from [2].

causalities among adjectives [2], [5]. Under either approach,panelists are subjected to stimuli and asked to provide
scores for individual or paired adjectives. However, panelists are generally not given specific instruction on how
quickly to respond or whether to do so during or after
experiencing the stimuli. As a result, such experiments do
not consider the time spent on the response and, therefore,
the resulting causal models do not consider the dynamic
aspects of perceptual/affective responses.
Human affective experiences change temporally fromthe moment that stimuli are perceived via the senses. For
example, the perceived sensory information regarding thetaste and flavor characteristics of food are perceived synthetically to create evaluations such as “satisfaction” or“like.” Such series of perceptual and affective responses
represent dynamic events expressed along the temporalaxis. Researchers have already made attempts to record thepattern of temporal change of sensation or affect. The timeintensity or the trace method, for example, has been used
to record the temporal change of intensity of single or two
types of sensations/affects [8], [9], [10], [11]. These methods

1949-3045 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TAFFC.2019.2942931, IEEE
Transactions on Affective Computing
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 2019

M ETHODS FOR C AUSALITY M ODELING

Of the many approaches that have been developed byresearchers to capture temporal change in the sensations
and affects, we selected the TDS method developed by [12],(b) Affective/Evaluative adjectives.

Fig. 2. Graphical user interfaces used in TDS experiments for recording(a) sensory and (b) affective/evaluative adjectives.
(a)
Stop

Start
<Sweet>0Panelist1

0

Rep.1

Start

(b)

0

1

0
0

1

<Sour>Stop
10

Panelist1 Rep.2

0

Rep.3

0

1

1
1

0

Rep.3

1

Normalized time [s]

Start
Rep.1

Rep.1 0Panelist 2 Rep.2

1
Stop

Normalized time [s]<Sour>
兟兟兟2

2.1 Measurement of Perceptual/Affective Responses:
Temporal Dominance of Sensations Method(a) Sensory adjectives.

兟兟兟

0

1

0

1
0

0

1

Accumulation for all panelists and repetitions(c)
Dominance rate [%]involve the use of a slider or joystick to continuously express the magnitude of subjective experiences. The temporal
dominance of sensations (TDS) method has been applied
to the measurement of simultaneous change in multiple
sensations/affects over a number of trials and subjects [12],[13], [14]. The details of the TDS method are describedin Sec. 2.1. Whereas the methods described above apply
sensory evaluation, physiological quanta can also be used
to predict the temporal change of responses. For instance,electroencephalographic data from the scalp [15], [16] andfacial expression readings [17] are both considered to reflect
temporally varying human subjective experiences. Research
involving such physiological quanta has centered on theconsistency between such quanta and subjective experience.
However, regardless of whether a subjective or physiological quanta approach is adopted, time series of perceptual/affective response are typically discussed qualitatively
or analyzed by focusing on changes in a single sensation,
while the temporal causalities between perception and affectare neglected.
In this study, a method of causality modeling using perceptual and affective information that dynamically changes
from the moment that a stimulus is experienced was developed. The approach is implemented by first recording
temporal changes in perceptual and affective responses asmodeling targets through the use of TDS methods. By
mathematically representing the resulting multivariate timeseries in the form of a vector auto-regression model, thecausalities among the time series can be investigated interms of their Granger causalities. The Granger causality,
explained in Sec. 2.3, is recognized when one variable
plays a significant role in predicting another. Finally, theresulting Granger causality model is contracted through theapplication of an information criterion. The causal models
established through this approach express how affects orpreferences temporally vary and can aid in the affective
design of products and the prediction of consumer affective
response.
In this study, food was used as the source of stimulus andperceptual and affective responses were measured from themoment of consumption of the food. In the present context,“perception” encompasses multimodal sensations including
taste, flavor (smell), and texture (touch). The primary reason
for using food as stimuli was that the protocol of the TDS
method was developed in the field of food products andhas been applied in a large number of foods. Nonetheless,the methods were also adapted to visual and auditory stimuli [18], [19]. However, they may require further researchin order to establish general protocols. Finally, the experimental data of the present study was also used in [20] toinvestigate the availability of vector auto-regressive (VAR)
model; however, the causal modeling was not discussed
there.

2

TDS curve for “Sour”
50

0

0

Normalized time [s]1Fig. 3. Generation of TDS curve. (a) Binary data obtained from a single
trial. (b) Binary data corresponding to repetition of selection of “sour”
by panelists. (c) TDS curve for “sour” generated by smoothing thedominance rate obtained from the binary data in (b).

which can efficiently measure multiple subjective attributes
from the moment that food is consumed.
The TDS method can be implemented using graphicaluser interfaces such as those shown in Fig. 2. A panelist
pushes the start button as soon as they have placed the foodin their mouth and then pushes the button corresponding
to their dominant sensation at that moment. Each time thedominant sensation changes, they push the corresponding
button, repeating the process until the food has been completely ingested, at which point they push the stop button.
They are allowed to push the same button as many times as
they desire.
The TDS method is used to produce a time series foreach sensation/emotion called the TDS curve, which shows
the dominance of the sensation relative to others. Fig. 3
illustrates the generation of the TDS curve. Fig. 3(a) showsbinary data obtained from a single trial representing the

1949-3045 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TAFFC.2019.2942931, IEEE
Transactions on Affective Computing
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 20193

time interval during which each sensation/emotion is selected. The horizontal axis represents the normalized timespan from the ingestion (start) to the swallowing of the food(stop). From the data accumulated from all panelists over
several trial repetitions, the number of selections of eachsensation/emotion at each time is counted (Fig. 3(b)). The
dominance rate is then computed by dividing the numberof selections for each button by the total number of trials (no.
of panelists × no. of repetitions) (Fig. 3(c)). For instance, thedominance rate during a time interval in which all of thepanelists select the same button will be 100%. Finally, a continuous TDS curve is obtained by smoothing the dominance
rate results.

(x, y, z ). The VAR (q ) model for estimating the value of x
at time t using z is expressed as follows:x̂t = c1 +

q
∑

(j)

a1 xt−j∆t +

j=1

q
∑

(j)

a2 zt−j∆t

(2)

j=1

Similarly, the VAR (q ) model for estimating the value of x
using y and z is expressed asx̂t = c2 +

q
q
∑
∑
(j)
(j)
b1 xt−j∆t +b2 yt−j∆t
j=1

j=1

q
∑
(j)
+
b3 zt−j∆t .

(3)

j=1

2.2 Mathematical Modeling of Multiple Time Series:
Vector Auto-regression Model
The data obtained using the TDS method (TDS curve) are
multivariate time series representing the relative intensity ofperception and affective responses for each evaluative word.
These time evolutions can be mathematically expressedusing a vector auto-regression (VAR) model. Once the timeseries are expressed in the form of a VAR model, the causalities among the variables can be statistically investigated byapplying the concept of Granger causality, explained in thefollowing section, and which is the main reason for the VAR
model being used by the proposed method. The VAR modelfor n variables with a model order q is expressed as follows:
yt = c + A(1) yt−∆t + A(2) yt−2∆t · · · + A(q) yt−q∆t .

(1)where yt is a vector of n variables, A(j) is the coefficientmatrix for the past j lags (A ∈ Rn×n ), c is a constant vector,
and n is the number of types of recorded sensations andaffective responses. At each time t, one of the n variables
is expressed as a linear function of the past q time-lagged
values of all variables, including itself. Each parameter ofthe VAR model (A(j) , c) is estimated using the least-squares
method. The model order q is selected based on how the
information criterion is to be managed; we adopted theorder at which the information criterion becomes minimum
over the range q =1–10. As the information criterion, weused the Akaike information criterion (AIC) [21], which is
explained in Appendix A. The AIC will be also used in thelatter process, as described in Sec. 2.4.3. Because the VAR
model uses discrete data, we discretized the continuous TDS
curves with a sampling period of ∆t.

The squared sums of estimation errors obtained by (2) and(3) are R0 and R1 , respectively. Then,

Fy→x =(R0 − R1 )
R1 /(T − nq − 1)

(4)

asymptotically follows a χ2 (n) distribution in which T is
the sample number and n is the number of variables. Wheny directly affects x, R0 > R1 holds and Fy→x > 0. In ourstudy, the presence of Granger causality from y to x could be
admitted when the p value of Fy→x was equal to or less than
0.10. In the presence of more than three variables, causalities
are judged in the same manner.
In the remainder of this paper, this conditional Granger
causality will simply be referred to as Granger causality.
The MVGC Matlab Toolbox [24] was used to conduct theGranger causality testing. Stationarity of data was assumed
in the causality testing process.
2.4 Causality ModelingIn this section, we explain the method used to establish
causal models through application of Granger causality andthe information criterion. We apply Granger causalities between sensory and affective adjectives in constructing a twolayered model. Following this, we analyze the causalities
among the affective adjectives. Finally, the AIC is applied to
the Granger causalities detected at each stage to determine
whether they will be retained in the final model. As an
example of this model construction, in Sec. 4, we establish amodel using the sensations and affects generated by eatingstrawberries.

2.3 Causality between Time Series: Granger Causality

2.4.1 Initial Two-layered Structure of Granger Causality

To represent the statistically judged causality between timeseries produced by the VAR model, the proposed method
applies the Granger causality approach [22]. To judge thecausality from variables y2 to y1 , two VAR models areapplied to estimate y1 —one that includes y2 and one that
does not. If, based on a comparison of errors, the estimation
accuracy of y1 is significantly increased through the inclusion of y2 , a Granger causality from y2 to y1 is accepted.
In this study, conditional Granger causality [23], which
represents an expansion of the original Granger causality
approach from two to multivariate causality, was applied.
Here, we explain how to test for conditional Granger
causality from y to x in the presence of three variables

Many earlier studies employed two-layered models withsensory and affective descriptors being at the lower andupper layers, respectively [6], [7], [25], [26]. In some cases,
preferential adjectives will be placed in layers higher thanthose in which other adjectives are placed [1], [2], [3],[27], [28]. Whereas the causal structure of the upper layer
is largely affected by individual differences [27], sensory
and affective adjectives can be semantically clearly segregated [26]. Based on these facts, the model can be expandedfrom a two-layer perception and emotion structure [4]. We
initiate the process by locating the sensory and affective
adjectives in the lower and upper layers, respectively, as
shown in Fig. 4(a), and then using this structure as a basis.

1949-3045 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TAFFC.2019.2942931, IEEE
Transactions on Affective Computing
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 20194

Positive
Negative
Positive
Negative

Positive
Negative

Positive
Negative

E

D

E

F

Affective/
Evaluative

D

E

F

D

E

F

D

E

F

D

A

B

C

Sensory

A

B

C

A

B

C

A

B

C

A

(a) Location of adjectives(b) Detection of GC between
below and upper layers(c) Initial two-layered model(d) Detection of GC among
upper layer

F

B

C

(e) Granger causality structure

Fig. 4. General flow for constructing the structure of Granger causality (GC); the model is developed from (a) to (e). (a) Sensory adjectives andaffective adjectives placed into the bottom and upper layers, respectively. (b) and (c) Initial two-layered model is constructed by investigating Granger
causalities between lower and upper layers. (d) Granger causalities among the adjectives in the upper layers. (e) Multi-layered structure is built forvisual clarity. Highlighted nodes in (b) and (d) are the target nodes investigated for Granger causality at each step.

As shown in Fig. 4(b), the Granger causalities among
all of the variables in the bottom layer (A, B, C ) and one
variable in the upper layer (D) are exhaustively investigated. In this example, a positive causality from B to C,
a bidirectionally positive causality from A to D, and anegative causality from C to B are detected2 . The positivities
and negativities of the respective causalities are determinedfrom the signs of the partial regression coefficients in the
VAR model. The arrows of causality in the figure show
time directionality; i.e., they represent how a causal variableaffects a resultant variable ∆t periods later. Performing
this operation repeatedly on each of the variables in theupper layer produces a two-layered model containing all
the causalities between perception and affect, as shown in
Fig. 4(c). In this study, we applied a 10% level of significancein the Granger causality test results as the criterion for the
adoption of a causality into the model. The model becomes
simpler as this criterion is made more severe, as doing so
reduces the number of detected causalities. In the example
of following sections, the adoption of 5% or 1% significancelevels led to overly simplified models; hence, we adopted
10%. Even if a high significance level is used, redundantcausalities are excluded through the procedure describedin Sec. 2.4.3. Therefore, in this step of the initial two-layer
modeling, it is better to include as many potential causalities
as possible. If the model is overly simplified at this stage,
the subsequent processes will not revitalize the missing
causalities.
Another factor that reduces the number of causalities
detected under conditional Granger treatment is increasing
the number of variables that are considered simultaneously.
This is because increasing the number of explanatory variables in the VAR model reduces the contribution of eachvariable to the objective variable, making the detection of
statistically significant contributors more difficult. Therefore, the proposed method exhaustively investigates thecausalities among a subset of the variables to establish amodel based on their superimposition, as described above.
By restricting the number of variables considered simultaneously in this manner, the model is prevented from becoming
too simple (See Appendix B).
2. Because unidirectional causalities between all considered variables
are investigated, causality from upper to lower variables (D → A) or
among lower variables (C → B ) can be detected.

2.4.2 Granger Causality in Upper Layers
The two-layered model includes causalities both among sensory variables and between sensory and affective variables.
In this subsection, Granger causalities among the variablesin the upper layer, which are not considered in the twolayered model, are exhaustively investigated. In the example
shown in Fig. 4(d), causalities from D and F to E are
detected as a result of the causality investigation. Variables
such as E , which are affected by other variables in boththe lower and upper layers, are expected to represent more
abstract concepts. Thus, by allocating E even at the upper
layer, the causal model in Fig. 4(e), which fully considers thebidirectional causalities between all variables, is obtained.
2.4.3 Finalization by Akaike Information Criterion
The causal model obtained thus far was established bysuperimposing the Granger causalities among subsets of all
variables. In the process, the estimation accuracy of the overall model was not considered, and it is possible that someredundant causalities with minimal contributions to theaccuracy of the overall model have been included. It is also
possible that some indirect causalities have been includedin the model. To address these issues, the model is modified
based on an information criterion, the AIC. The AIC is
an index reflecting both the estimation accuracy and thecomplexity of the model. We sequentially applied the AIC
to exclude redundant causalities to produce a simplified
model. For example, if the removal of a causality decreases
the AIC, then the corresponding arc is excluded from themodel. This operation is repeated until the AIC ceases todiminish upon further causality exclusion, at which point
the finalized contracted model is obtained. Note that there
is no guarantee that the final causal model produced by thismethod is the optimal solution.

3

TDS E XPERIMENT U SING S TRAWBERRY

To test the structure defined in the preceding section, we
conducted a TDS experiment using strawberries as a foodexample. In the following subsections, we describe how
a causal model was established based on time series ofperceptual and affective responses obtained by eating strawberries.

1949-3045 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TAFFC.2019.2942931, IEEE
Transactions on Affective Computing
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 2019

5

TABLE 1
Sensory and affective/evaluative adjectives used in strawberry experiment. Japanese words presented to the panelists are shown in Italics.

Sensory
Sweet
WateryRefreshing
Juicy
Sour
Soft
Melty

Panelists

The panelists were ten Japanese university students (eightmale and two female, age: 22–24), all of whom provided
written informed consent before the experiment. They were
not studying affective sciences or engineering. None of thepanelists routinely smoked, and consumed taste-altering
substances such as coffee one hour before experiment.
3.3

Tasks

The TDS experiments were conducted using a touch panel.
Before the experiments began, the panelists were allowed
to operate the interface to get used to the task of pushingbuttons and the use of the touch panel.
One strawberry was used per one trial. After drinking water to rinse their mouth at the beginning of eachtrial, each panelist took the strawberry behind a partition,
pinched their nose and, with the other hand, placed theDominance rate [%]Selection of Adjectives

Some researchers have pointed out that displaying too
many words on the interface overloads the panelists of
TDS methods. As a suggested limit, the number of words
should not exceed ten [29]. We selected the adjectives to be
used through a preliminary survey in which a check-typed
questionnaire generated by the authors was administered.
The questionnaire presented a total of 137 adjectives
expressing characteristics of strawberries that are perceived
via taste/smell/texture and characteristics representing theaffective/evaluative aspects of strawberry consumption.
The sensory adjectives included words expressing basic
taste factors such as “sweet” and “sour” and words pertaining to flavor and texture such as “fruity” or “refreshing”
to represent a cooling sensation in the mouth. For affective/evaluative adjectives, we added some words to the listof adjectives identified by King et al. [30].
Seven panelists participated the preliminary experimentby checking all adjectives that applied to their perceived
sensations or affects while eating strawberries. Based on
the total number of selections, we identified seven sensory
and affective/evaluative adjectives apiece. Table 1 lists the
fourteen selected adjectives by type. We avoided the useof compositive words and attempted to use words that
represented multiple synonyms. We also attempted to select multidimensional sensations without redundancy andomission. Prior to the actual experiment, we presented
definitions of each word to the panelists to standardize theirinterpretations.
3.2

Affective/Evaluative
Like
suki
hozyunna
Flavorous
Freshsinsenna
Happy/Satisfied kohukuna/manzokuna
Delicious
oishiiNaturalsizennaElegant
johinna

100

Sweet

80

Sour

WaterySoft

Refreshing
Melty

Juicy

60
40
20
0
0

Dominance rate [%]

3.1

amai
mizuppoisawayaka/sappari
jusinasuppaiyawarakai
torokeruyona/namerakana

100
80

0.5
Normalized time(a) Change of dominance for sensory adjectives.
Like

Flavorous

Fresh

Delicious

NaturalElegant

1

Happy/Satisfied

60
40
20

h

0

1
0.5
Normalized time(b) Change of dominance for affective/evaluative adjectives.

0Fig. 5. Temporal change in perceptual and affective/evaluative responses. TDS curves for (a) sensory adjectives and (b) affective/evaluative adjectives.

strawberry into their mouth with their eyes closed. This procedure was used to cut off sight and smell stimuli prior toeating. Immediately after putting the strawberry into theirmouth in one bite, each subject opened their eyes, released
their hands from their nose, and began the task of pushingbuttons. This task continued until the food diminished in
the participants’ mouth and the stop button was pressed.
Testing involved alternately displaying selections of eachtype of adjective on the interface, as shown in Fig. 2, fromwhich the panelist made a selection. The order in which two
types of adjectives were presented, as well as the location of
the adjectives on the interface, were randomized for eachpanelist. Each panelist conducted this task three times foreach type of adjective group, completing six trials in total.
Each trial took approximately 30 s, although the duration of
the task was not set. Also, for any of the TDS procedures,the panelist could press the buttons as frequently as s/hewanted. A break of a few minutes was given between trials.

1949-3045 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TAFFC.2019.2942931, IEEE
Transactions on Affective Computing

Dominance rate [%]JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 2019100

TDS curve of juicyTDS curve

50
0

6

0.5
Normalized time

0

1

Change ofdominance rate [%]Differentiation
Differenced TDS curve
10
0
-10
0

0.5
Normalized time1

Fig. 6. Example of differentiation of TDS curve. Each curve is differentiated using the sampling period ∆t.

3.4 Results
The graphs in Fig. 5 show the perceptual and affective
responses obtained by the experiment. The horizontal axis
represents the normalized time span from the intake ofthe strawberry until the completion of each perception trial(average: approximately 30 s). A moving average filter witha range of 1.5 s was used to smooth the dominance rate3 . The
sensory adjectives “juicy” and “sweet” were most common
in the first halves of the trials, while “sour” was prominent
in the second halves. The affective/evaluative adjective “delicious” was dominant overall, although “flavorous” wasprominent in the first halves and “like” and “fresh” wereprominent in the second halves. By contrast, the adjectives“soft,” “melty,” “elegant,” “happy/satisfied,” and “natural”
were infrequently selected throughout the experiment. As
these sensations were not considered to be dominant basedon the criterion in [12], that is, whether the responses are
significantly greater than the chance, they were excluded in
the subsequent construction of the causal model.

4of 1 s to descritize the continuous data obtained from theexperiment.
As the Granger causality test assumes data stationarity,
we investigated the stationarity of the time series obtained
from the experiment by conducting an Augmented DickeyFuller (ADF) test [31] as a unit root test, from which it wasdetermined that the possibility of non-stationarity could not
be denied for some of the variables. We therefore de-trended
the TDS curves by translating them to difference series
using a sampling period of ∆t. Fig. 6 shows an exampleof the translation. Subsequent ADF testing on the resulting
difference series denied the non-stationarity of the data at
a significance level of 5% for all variables. This differencetime series was therefore used to establish the causal model.
The translation of causality from time to difference series
ensured that changes in a causal variable would result inchanges in its resultant variables.
Finally, we adopted q = 1 as the order of the VAR model
following the criterion described in Sec. 2.2.
4.2 Initial Two-layered Model
Fig. 7(a) shows the initial two-layered model constructedusing the method explained in Sec. 2.4. The Granger causality was tested for among the five sensory adjectives in thebottom layer and one of the four affective/evaluative variables in the upper layer. The following causalities between
sensory and affective/evaluative adjectives were identified:strawberry sweetness causes flavorous; bidirectional causality
was detected between “sweet” and “delicious”; as waterystrawberries are perceived as having a little taste, negative
causalities between this sensory adjective and “fresh” and“like” were observed. Among the sensory adjectives, bidirectional causalities were detected among “sweet,” “juicy,”and “refreshing.” Each node of the graph contains thecorrelation coefficients between the values observed on the
TDS curves and the values estimated by the VAR models.
The VAR model for each adjective includes the adjective
and all adjectives that are linked to it through Granger
causality as explanatory variables. For example, in Fig. 7(a),the VAR model for “like” includes “watery” and “like”
itself as explanatory variables. If an adjective was linked tono other adjective through Granger causality, its value wasestimated using only its past information (AR model).

C ONSTRUCTION OF C AUSAL S TRUCTURE WHILEEATING STRAWBERRY

4.3 Model with Full Granger Causalities

We constructed a causal model of perceptual and affective responses for strawberries by applying the method
explained in Sec. 2.4.

Fig. 7(b) shows the modified causal model showing causalities among the affective/evaluative adjectives. Here, positive causalities from “delicious” to “flavorous” and from“fresh” and “flavorous” to “like” have been newly detected.
Ultimately, only the adjective “like” was found not to cause
other sensations and was therefore placed in the top layer.
The resulting model shows the temporal process in which
the taste or texture of strawberry causes an affect and thena preferential judgment. For instance, both “sweet” and“delicious” lead to “like” via “flavorous.” Although thiscausal structure of perception-emotion-preference has beenreported in previous studies [1], [2], [3], [4], [5], [27], [28],our model suggests that these previously obtained causal
structures can be observed as actual dynamic events startingfrom the moment of experiencing a stimulus.

4.1 Data Processing
The sampling period of the VAR model, ∆t, defines theminimum response speed considered under the model.
During our experiments, the time from the selection of abutton to the selection of another button was approximately
1 s at minimum; accordingly, we used a sampling period
3. Although the range of the filter did not significantly affect themodel that was ultimately obtained, the estimation accuracy of themodel increased at longer ranges because doing so suppressed difficultto-predict high frequency content.

1949-3045 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TAFFC.2019.2942931, IEEE
Transactions on Affective Computing
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 20197

Positive
Negative(0.69)( ) Correlation coefficients between
observed and estimated values(0.66)

Fresh(0.68)

Like(0.35)(0.50)

Watery(0.75)

Flavorous(0.66)

Refreshing

Affective/
Evaluative

Delicious(0.66)(0.84)

Juicy

Sour

Sweet

Sensory(a) Initial two-layered model.
(0.69)Like< AIC: -848 >Evaluative(0.69)(0.69)
Fresh

Flavorous(0.75)

Affective

Delicious(0.50)(0.35)
Watery(0.66)(0.66)(0.84)

Juicy

Refreshing

Sour

Sweet

Sensory(b) Model of full Granger causalities.

(0.68)
Like(0.69)Fresh< AIC: -850 >

.32

-.99

(0.68)(0.35)
Watery(0.75)

Flavorous-.85(0.50) .19
Refreshing(0.66)
Juicy.38

Affective

Delicious.43 .51(0.68).24.53

Evaluative

Sweet(0.66)
Sour

Sensory.18(c) Final reduced model.
Fig. 7. Structure of Granger causality of adjectives for strawberry. (a) Initial two-layered structure comprising sensory and affective/evaluative layers.
(b) Model of full Granger causalities. (c) Final reduced model. Values aside arcs are the coefficients of influence.

4.4

Final Reduced Model

Fig. 7(c) shows the final AIC-based reduced model, in
which four causalities have been excluded from the model
shown in Fig. 7(b). The excluded causalities are those from“delicious” to “flavorous,” from “juicy” to “sweet,” from“sour” to “sweet,” and from “fresh” to “like.” As a resultof the contraction process, the AIC of the model decreases
from -848 to -850. Under the reduced model, the causal
relationships among the adjectives in the lower layer are
simplified and “sour” is decoupled from other adjectives;
nevertheless, the estimation accuracy of the original modelis retained. For example, an examination of the correlation
coefficients between the observed and estimated values reveals that “like” and “flavorous” can be estimated efficiently
using fewer variables while maintaining accuracy, as thecorrelation coefficients representing the estimation accuracyof these variables are reduced by an insignificant amountunder simplification even though there is one less causal(explanatory) variable. This demonstrates how simplification through application of the AIC to trim the set of
explanatory variables produced by the VAR model enables
a more efficient expression of the model.

TABLE 2
Results of cross validation using sample groups A and B. Correlation
coefficients R between observed and estimated values are listed.

Adjectives
Like
Delicious
Fresh
Flavorous

5

Cross validation(test data, training data)(A, B) (B, A) Average
0.52
0.44
0.35
0.43
0.68
0.56
0.61
0.56
0.59
0.67
0.30
0.49

Original model
based onall samples
0.68
0.75
0.69
0.68

VALIDATION OF C AUSAL M ODELWe then investigated the quantitative and semantic validityof the proposed method of causal modeling by performing
cross validation to assess the generalization ability of theestimation model for affective responses constructed basedon the causal model. We further investigated the semantic
validity of our causal model obtained via a comparison
experiment using fake models.

1949-3045 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TAFFC.2019.2942931, IEEE
Transactions on Affective Computing
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 2019

5.1 Quantitative Validity
A two-fold cross validation approach was used to validate
the established model. The performance of the estimation
model was assessed by dividing the total sample set into
two groups, which were used as sources of test and training data, respectively. The estimation operation was thenapplied to the two sample groups alternatively to test
the generalization ability of the model. TDS curves weregenerated from each sample group (groups A and B, each
representing the results obtained by five panelists over three
trials). Causal models were then established by applying
Granger causality and the AIC, with VAR models constructed for each sample group to estimate their respective
affective/evaluative variables. Estimates were constructed
for one of the sample groups using the estimation equations
obtained from the other group and, finally, correlation coefficients between the observed and estimated values were
calculated.
Table 2 shows the results of the cross validation. The
largest and smallest mean correlation coefficients were obtained for “fresh” (0.59) and “like” (0.44). The mean correlation coefficient among all of the variables was 0.52. Relative
to the original model established from all of the samples, thecorrelation coefficients decreased by 0.18 on average, a resultthat may be partially attributable to individual differencesamong the panelists.
5.2 Semantic Validity
In addition to quantitative consistency, we assessed thesemantic reasonableness of the results produced by our
method. The model shown in Fig. 7(c) expresses the process
through which the taste and the texture of a strawberry influences affect and preference. It is seen that “sweet” evokes“delicious” and “flavorous” sensations and that “flavorous”
sensation leads to a judgment of “like.” Perception of “watery” reduced the sensation of “fresh” and “like.” Although
all of these sensations appear to be semantically reasonable,
the semantic validity of a model requires support by the
judgments of multiple individuals.
Accordingly, we presented the four distinct causal models shown in Fig. 8 to a set of panelists and instructed
them to rank the models from the viewpoint of semantic
validity. The model in Fig. 8(a) is identical to the one
in Fig. 7(c). We generated three comparative models byadding slight modifications from the model (Fig. 8(b)–(d)).
Even though we had no comparative models as immediatereferences, to avoid presenting completely random models
we generated the three comparative models by excluding
one causality each from the model in Fig. 7(c) and thenadding a new causality. In the model in Fig. 8(b), we haveexcluded the causality from “watery” to “like” and added
one from “sour” to “like.” Similarly, for models (c) and (d)we have changed the cause of “flavorous” from “sweet” to“juicy” and the cause of “like” from “flavorous” to “fresh,”respectively.
Sixteen university students participated in this experiment (fourteen males and two females, age: 22–25). The
panelists compared the validity of the causalities includedin each model and ranked the models while eating strawberries.

8

TABLE 3
Ranking of four causal models. The total ranks at the bottom were
determined by summing the ranks awarded by all participants for eachmodel.

Panelists
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Rank sum
Total(a) Original
2
2
1
3
2
2
4
1
2
1
2
3
2
4
3
2
36
2

Rank
(b) M I (c) M II
4
1
3
1
2
3
2
4
3
1
1
3
1
2
3
2
1
4
2
3
4
3
2
1
3
1
3
1
1
2
1
3
36
34
2
1

(d) M III
3
4
4
1
4
4
3
4
3
4
1
4
4
2
4
4
53
4

The semantic validity of the model obtained using the
proposed method would be considered to be validated if
the rank given to the model in Fig. 8(a) was not below thosegiven to the three similarly structured comparison models.
Table 3 lists the results of the ranking for each panelist.
The bottom total rank (rank sum) is calculated as the sumof the ranks given by all panelists for each model. The rank
sum of modified model (d) is largest, while those of (a), (b),and (c) are of similar magnitude. The results of Friedman
testing suggest a significant difference among these ranks ofχ2 (3) = 8.47. Similarly, the results of post-hoc Wilcoxon’s
rank sum testing reveal that there is a significant difference
at a level of 1% between model (d) and the other threemodels. In other words, the semantic validity of (d) is lower
than those of (a), (b), and (c) but there is no significant
difference among the latter three models. However, thesemantic validity of the causal model obtained using the
proposed method is not lower than those of the other causal
models to any significant degree.

6

D ISCUSSION

The results described in the preceding section support thequantitative and semantic validity of the causal model established by our method. In terms of quantitative validity,
the average correlation coefficient between the observed affective response values and those estimated from the model
was approximately 0.7. However, this value was decreased
to 0.52 in the cross validation. The semantic validity of acausal model is generally determined by panelists; basedon a vote of the sixteen public panelists in the semantic
validation exercise, the established model was judged to besemantically valid, and no marked semantic contradictions
are apparent in the model shown in Fig. 7(c).

1949-3045 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TAFFC.2019.2942931, IEEE
Transactions on Affective Computing
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 20199

Positive

Negative

Like
Fresh

WaterySour

Refreshing

Like
Flavorous

Delicious

Fresh

Juicy

Sweet

Watery(a) Established model

Sour

WaterySour

Refreshing

Delicious

Juicy

Sweet

(b) Modified model I (M I)

Like
Fresh

Refreshing

Flavorous

Like
Flavorous

Delicious

Fresh

Juicy

Sweet

Watery(c) Modified model II (M II)

Sour

Refreshing

Flavorous

Delicious

Juicy

Sweet

(d) Modified model III (M III)

Fig. 8. Comparison of four models to assess semantic validity. (a) Model obtained using proposed method. (b)–(d) Models in which one arc isremoved from (a) and one arc is added.

It is interesting to note that a bidirectional causality
was detected between “delicious” and “sweet” in the established model. There are two potential interpretations of
this relationship. First, it might be the case that “sweet” and“delicious” are semantically similar. Our model indicatedthat sweet strawberries are delicious, i.e., that the terms
are semantically close. Another interpretation is that there
is a recurrent process between perceptual and affective
response. When an individual judges a sweet strawberry to
be delicious, he/she may be judging a delicious strawberry
to be sweet at the same time. Although the causalities
from perceptions to affects are intuitive, inverse causalities
can also occur. For example, it is well known that dual
relationships hold between emotion and pain [32], [33]. In
our model, the sensation of sweetness at a certain moment
causes a feeling of deliciousness ∆t periods later. In a similarmanner, a sensation of deliciousness causes an experience ofsweetness ∆t periods later.
One promising use of the newly established model isthe prediction of temporal changes in affective responses.
For instance, it allows us to estimate the propagation of atransient change in one gustatory feeling. If the sweetness
experienced from strawberries is more intense at any one
instant, then Fig. 7(c) indicates that the “flavorous” response
increases ∆t later, and the “like” response also does so after
another ∆t. “Sweet” also influences its own intensity inthe next several periods of ∆t. In this way, the change in
one response leads to changes in multiple other responses
during finite intervals. These subsequent changes, caused bya change in one variable, are analyzed based on the impulse
responses for the VAR models. This kind of analysis is more
effective for processed foods and cuisines than for fruits, inthat it supports the design or manipulation of the affective
experiences produced by tastes, flavors, and textures.
Although in this study, perceptual and affective re-sponses were estimated using the VAR model, which islinear and time-invariant, in the future it should be possible
to apply nonlinear models or modeling that includes temporal change in causal structures. The human perceptual andaffective responses are nonlinear [34] and our linear method
therefore only approximates them. The concept of Granger
causality can also be applied to nonlinear models [35]. As
suggested by the profile of TDS curves obtained from ourexperimental results, the causal structure of perception andaffect is perhaps time-varying. For instance, sensations suchas “watery” were mostly observed at the end of the eatingprocess and were not selected beforehand. Thus, if the
proposed method is applied to portions of the overall TDS
curve (e.g., the first or second half only), the resulting causal
models might differ from those we obtained. This follows
Lepage et al. [36] where the entire period was split into a
few phases and analyzed. Capturing the temporal change
in the causal structure itself can therefore uncover affective
dynamics.
The results of the cross-validation exercise suggest thatthe generalization ability of the proposed model for predicting affective responses is somewhat lower than that of an
independent sample group. One of the main reasons forthis lies in individual differences in terms of perceptualand affective response. In cases in which such individual
differences are important, it can be effective to classify
panelists using cluster analysis [27]. By doing so, causal
models can be produced for each classified group and themodel suitable to the characteristics of the target consumers
can be selected. It has also been reported that individual
differences in eating manner affect the dynamic change ofsensed texture [37]. To reflect and account for individual
differences in eating manner, panelists can be screened bytime spent on eating or bite/swallowing timing [17], [38].

1949-3045 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TAFFC.2019.2942931, IEEE
Transactions on Affective Computing
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 201910

The VAR models use all variables that are associated withothers through Granger causality, along with the variables
themselves, as explanatory variables. For example, the VAR
model to estimate the values of D (yD ) is expressed in termsof the explanatory variables yA , yB , and yD as follows:

E
D
A

B

C

ŷD,t = c +Fig. 9. Example of the structure of Granger causality.

C ONCLUSION

q
∑

q
∑

(j)

aBD yB,t−j∆t

j=1

(6)

(j)
aDD yD,t−j∆t .

j=1

The objective of this study was to develop a method forthe causality modeling of perception and emotion using
dynamic perceptual and affective information. We recorded
perceptual and affective responses starting from the beginning of the process of consuming food and expressed the resulting time series using VAR models. We then investigatedthe Granger causalities among the subjective responses toestablish a causal model of perception and affect. Thismodel was further simplified through the application of an
information criterion (AIC) to construct a final simple andaccurate model. Testing of the constructed causal model
via the sensations obtained in the process of consuming a
strawberry as an example revealed a coherent perceptionemotion-preference causal structure. The model was further
assessed through cross validation and comparison withmodified models. The results revealed that, although thegeneralization abilities of the proposed model in estimating affective response were somewhat low, the model was
semantically reasonable. The proposed method should be
useful in the affective design of products and in the prediction of consumers’ affective responses and should be ofuse in analysis of affective dynamics. In future research, thevalidity of the method in application to a variety of foods
should be further assessed.

A PPENDIX A
A KAIKE I NFORMATION C RITERION
We evaluated the validity of the overall causal model interms of the typical information criterion AIC [21], which isexpressed as follows:

AIC = −2L(θ) + 2k(j)

aAD yA,t−j∆t +

j=1

+

7

q
∑

The variables with no associated causal variables, such as
yA , yB , and yC , are considered to be exogenous variables
and are not taken into account in the computation of the
AIC. Then, assuming that the errors (ϵi = [ϵD,i , ϵE,i ]T ) between the i th observed values of time series discretized intom samples (yi = [yD,i , yE,i ]T ) and estimated values by VAR
models (ŷi = [ŷD,i , ŷE,i ]T ) follow normal distributions, thejoint probability of the errors is expressed as follows:f (ϵi ) = √

1

2π

n√

|Σ|

1
exp(− ϵTi Σ−1 ϵi ),2(7)where Σ is the covariance matrix of the errors and n is
the number of estimated parameters, which is two in thiscase. The log likelihood function can then be obtained bymultiplying (7) over the entire interval as follows:

L(θ) = logm∏

f (ϵi ).

(8)i=1

Next, the number of estimated parameters, k , is obtained
as the sum of the number of parameters included in theestimation equations for each variable. For the example
shown in Fig. 9, the number of estimated parameters for
D is, from (6), 1 + 3q , while the number for E is 1 + 4q .
Therefore, k = 2 + 7q .

A PPENDIX BOVERLY S IMPLIFIED M ODEL E STABLISHED BY
C ONDITIONAL G RANGER C AUSALITIES AMONG A LL
VARIABLES(5)

where the θ are the parameters estimated by the model,
L(θ) is the log likelihood function obtained by estimating
the parameters using the maximum likelihood estimation
method, and k is the number of estimated parameters.
The first term on the right-hand side of the formula is
the reward term for goodness of fit of the model, whichbecomes smaller under more accurate models. The second
term represents the penalty term for the number of estimated parameters, k , and increases under more complex
models. Smaller AICs correspond to better models thatbalance accuracy and complexity well.
We illustrate how to compute the AIC of a causal modelby using the example shown in Fig. 9, which indicatesthe causalities among five variables (A–E) using arrows.
First, the log likelihood function is computed by using thevalues estimated using the VAR model for each variable.

The model shown in Fig. 7(b) was established by superimposing fourteen causalities among the subsets of thevariables. If, instead, the conditional Granger causality wereobtained by investigating all of variables simultaneously,
the causal model shown in Fig. 10 would be obtained. Thismodel includes only five causalities, and “refreshing,” “flavorous,” and “delicious” have become isolated adjectives. In
the presence of a large number of variables, as occurs in thiscase, conditional Granger causality cannot detect causalities
very well, resulting in overly simple models.

ACKNOWLEDGMENTS
This study was in part supported by JSPS Kakenhi(17K20002).

1949-3045 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TAFFC.2019.2942931, IEEE
Transactions on Affective Computing
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 201911

Positive ( ) Correlation coefficients between
observed and estimated values
Negative(0.69)
Fresh(0.35)
Watery(0.66)(0.55)

Like(0.38)
Refreshing(0.60)

Flavorous(0.63)
Juicy

Affective/
Evaluative

Delicious(0.74)
Sweet(0.66)
Sour

Sensory

Fig. 10. Overly simplified structure of Granger causality constructed by investigating conditional Granger causality among all adjectives at once.

R EFERENCES
X. Chen et al., “Materials’ tactile testing and characterisation for
consumer products
affective packaging design,” Materials &Design, vol. 30, no. 10, pp. 4299–4310, 2009.
[2] H. Nagano, S. Okamoto, and Y. Yamada, “Modeling semanticallymultilayered affective and psychophysical responses toward tactile textures,” IEEE Transactions on Haptics, 2018.
[3] K. Kidoma et al., “Graphical modeling method of texture-related
affective and perceptual responses,” International Journal of Affective Engineering, vol. 16, no. 1, pp. 27–36, 2017.
[4] S. Okamoto et al., “Layered-modeling of affective and sensory experiences using structural equation modeling: Touch experiencesof plastic surfaces as an example,” IEEE Transactions on Affective
Computing, vol. 14, no. 8, 2018.
[5] H. Nagano, S. Okamoto, and Y. Yamada, “Semantically layered
structure of tactile textures,” International Conference on Human
Haptic Sensing and Touch Enabled Computer Applications, pp. 3–9.
Springer, 2014.
[6] K. Drewing et al., “Systematic relations between affective and sensory material dimensions in touch,” IEEE Transactions on Haptics,
vol. 11, no. 4, pp. 611–612, 2018.
[7] W. Fujisaki, M. Tokita, and K. Kariya, “Perception of the material
properties of wood based on vision, audition, and touch,” Visionresearch, vol. 109, pp. 185–200, 2015.
[8] W. E. Lee and M. Pangborn, “Time-intensity: The temporal aspectsof sensory perception,” Food Technology, pp. 71–78, 1986.
[9] L. Duizer, E. Gullett, and C. Findlay, “Time-intensity methodology
for beef tenderness perception,” Journal of Food Science, vol. 58,
no. 5, pp. 943–947, 1993.
[10] D. E. Taylor and R. M. Pangborn, “Temporal aspects of hedonic
responses 1,” Journal of Sensory Studies, vol. 4, no. 4, pp. 241–247,
1990.
[11] E. Schubert, “Real time cognitive response recording,” Proceedingsof the Inaugural International Conference on Music Communication
Science, pp. 139–142, 2007.
[12] N. Pineau et al., “Temporal dominance of sensations: Construction of the tds curves and comparison with time–intensity,” Food
Quality and Preference, vol. 20, no. 6, pp. 450–455, 2009.
[13] G. Jager et al., “Temporal dominance of emotions: Measuring
dynamics of food-related emotions during consumption,” Food
Quality and Preference, vol. 37, pp. 87–99, 2014.
[14] P. Schlich, “Temporal dominance of sensations (TDS): a new deal
for temporal sensory analysis,” Current Opinion in Food Science,
vol. 15, pp. 38–42, 2017.
[15] R. Jenke, A. Peer, and M. Buss, “Feature extraction and selection
for emotion recognition from EEG,” IEEE Transactions on Affective
Computing, vol. 5, no. 3, pp. 327–339, 2014.
[16] S. Koelstra et al., “DEAP: A database for emotion analysis; using
physiological signals,” IEEE Transactions on Affective Computing,vol. 3, no. 1, pp. 18–31, 2012.
[17] D. Rocha-Parra et al., “Application of hedonic dynamics using
multiple-sip temporal-liking and facial expression for evaluationof a new beverage,” Food Quality and Preference, vol. 52, pp. 153–
159, 2016.
[18] C. Peltier, M. Visalli, and A. Thomas, “Using temporal dominanceof emotions at home. impact of coffee advertisements on consumers behavior and methodological perspectives,” Food Quality
and Preference, vol. 71, pp. 311–319, 2019.
[1][19] T. C. Merlo et al., “Measuring dynamics of emotions evoked by
the packaging colour of hamburgers using temporal dominanceof emotions (TDE),” Food Research International, 2018. [Online].
Available: https://doi.org/10.1016/j.foodres.2018.08.007[20] T. Okada et al., “Vector auto-regression model of temporal perceptual and affective responses towards food,” Proceedings of IEEE
Global Conference on Life Sciences and Technologies, pp. 43–45, 2019.
[21] H. Akaike, “Information theory and an extension of the maximum
likelihood principle,” Selected papers of hirotugu akaike, pp. 199–213,
1998.
[22] C. W. Granger, “Testing for causality: a personal viewpoint,”
Journal of Economic Dynamics and control, vol. 2, pp. 329–352, 1980.
[23] J. Geweke, “Measures of conditional linear dependence and feedback between time series,” Journal of the American Statistical Association, vol. 79, no. 388, pp. 907–915, 1984.
[24] L. Barnett and A. K. Seth, “The MVGC multivariate granger
causality toolbox: A new approach to granger-causal inference,”
Journal of Neuroscience Methods, vol. 223, pp. 50–68, 2014.
[25] T. Matsuoka et al., “Predicting texture image of covering fabricfor car seat by physical properties,” Journal of Textile Engineering,
vol. 54, no. 3, pp. 63–74, 2008.
[26] S. Guest et al., “The development and validation of sensory andemotional scales of touch perception,” Attention, Perception, &Psychophysics, vol. 73, no. 2, pp. 531–550, 2011.
[27] S. Okamoto et al., “Specification of individuality in causal relationships among texture-related attributes, emotions, and preferences,” International Journal of Affective Engineering, vol. 15, no. 1,
pp. 11–19, 2016.
[28] I. H. M. Hashim et al., “Tactile evaluation feedback system
for multi-layered structure inspired by human tactile perception
mechanism,” Sensors, vol. 17, p. 2601, 2017.
[29] N. Pineau et al., “Temporal dominance of sensations: What is agood attribute list?” Food Quality and Preference, vol. 26, no. 2, pp.
159–165, 2012.
[30] S. C. King and H. L. Meiselman, “Development of a method tomeasure consumer emotions associated with foods,” Food Quality
and Preference, vol. 21, no. 2, pp. 168–177, 2010.
[31] S. E. Said and D. A. Dickey, “Testing for unit roots in
autoregressive-moving average models of unknown order,”Biometrika, vol. 71, no. 3, pp. 599–607, 1984.
[32] R. A. Martin, “Is laughter the best medicine? Humor, laughter, andphysical health,” Current Directions in Psychological Science, vol. 11,
no. 6, pp. 216–220, 2002.
[33] S. Bruehl, C. R. Carlson, and J. A. McCubbin, “Two brief interventions for acute pain,” Pain, vol. 54, no. 1, pp. 29–36, 1993.
[34] F. Camargo and B. Henson, “Aligning affective responses withfabric features of vehicle seat: An approach using the Rasch
measurement model,” International Journal of Affective Engineering,
vol. 14, no. 3, pp. 193–202, 2015.
[35] E. Baek and W. Brock, “A general test for nonlinear granger
causality: Bivariate model,” Iowa State University and University of
Wisconsin at Madison Working Paper, 1992.
[36] M. Lepage et al., “Panel performance for temporal dominanceof sensations,” Food Quality and Preference, vol. 38, pp. 24– 29, 2014. [Online]. Available: http://www.sciencedirect.com/
science/article/pii/S0950329314000780[37] M. D. de Lavergne et al., “Eating behaviour explains differences
between individuals in dynamic texture perception of sausages,”
Food Quality and Preference, vol. 41, pp. 189–200, 2015.

1949-3045 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TAFFC.2019.2942931, IEEE
Transactions on Affective Computing
JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, APRIL 201912[38] S. Pecore, C. Rathjen-Nowak, and T. Tamminen, “Temporal orderof sensations,” Proceedings of 9th Pangborn Sensory Science Symposium, pp. 4–8, 2011.

Takumu Okada is a mechanical engineer who
received BS and MS degrees in engineeringfrom Nagoya University in 2017 and 2019, respectively. His research interests include affective engineering and haptics.

Shogo Okamoto received MS and PhD degrees
in information sciences in 2007 and 2010, respectively, from the Graduate School of Information Sciences, Tohoku University. Since 2010,
he has been with the Graduate School of Engineering, Nagoya University. His research interest
includes haptics, assistive technology, and affective engineering.

Yoji Yamada received a PhD degree from theTokyo Institute of Technology in 1990. In 2008,
he moved to Nagoya University, as a professor
after being an associate professor at the Toyota
Technological Institute and a group leader at
AIST. His expertise include robotics and safetyof machinery.

1949-3045 (c) 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.