JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015

1

Dynamic State-Space Modeling with
Factorial Memories in Temporal Dominance of
Sensations, Emotions and Temporal Liking
Kyoichi Tachi and Shogo Okamoto, Member, IEEE
Abstract—There are few mathematical models that describe how human perceptual and affective responses evolve with time. We
developed a state-space model of the interrelationships based on time-evolving perceptual and affective responses acquired by the
temporal dominance (TD) method. We deﬁned and computed the state variables that endow the system with memory using canonicalvariate analysis. Furthermore, we determined the model parameters on the basis of a cross-validation approach such that theobserved and estimated changes in the affective responses are highly correlated. We applied this method to the TD curves of
responses to eating strawberries and plum pickles, reported in our previous work. The model describes what happens during foodintake in terms of a few state variables that summarize a large number of observable responses reductively, thus helping to make theperceptual and affective dynamics understandable.
Index Terms—Temporal dominance, latent variable, memory, sensation, emotion, canonical variate analysis.

F

1

I NTRODUCTION

H

UMAN perceptual and affective responses to stimuli
are not static, but change dynamically with time.
Temporal-dominance (TD) methods [1], [2], [3], [4], [5] areused in food industry to record such multiple subjective
experiences in time-series form. This sensory-evaluation
technique enables the study of how perceptual (gustatory,
osmatic, and textural), affective, and preferential responses
evolve during food intake. TD methods have a great potential for new applications or experimental designs. For
example, TD methods realize research on dynamic linkage
among multiple types of feelings and human physiological
data such as skin conductance [4].
However, since TD has been in intensive use for acomparatively short time, only a few mathematical approaches to the data acquired by it have been developed.
These include methods based on Granger causality [6], [7],which seek causal relationships between the perceptual andaffective responses on the basis of their time-series; Markov
models [8], [9], [10], which treats TD data stochastically andcompute the expected probabilities that a type of subjective response is observed after another type of subjective
response; and principal motion analysis [11], which decomposes time-series TD responses into modal spectra and canbe used for discriminating between food products; a graphtheoretical model [12] to suggest the frequency of temporaltransition from one feeling to another; a covariance selection
model established by concurrent changes among multiple
types of feelings [13]. Visalli et al. proposed to separate theentire duration of stimulus experience into three phases andseparately analyze each phase [14]. Most other studies havecomputed static or quasi-static indices from the records of•

K. Tachi is with the Department of Mechanical Systems, Nagoya University, Japan. S. Okamoto is with the Department of Computer Sciences,
Tokyo Metropolitan University, Japan.
E-mail: okamotos@tmu.ac.jp

TD tasks; standard methods to model the dynamic nature of
TD response have yet to be established.
In a typical TD task (described in Section 2), approximately 10 types of responses are assessed. As TD tasks forrecording perceptual and affective responses are generally
conducted separately, up to 20 types of responses must
be explained in a single model. Such a large number ofobserved variables prevents intuitive understanding of therelationships within the data. A more reductive methodology is required if the interdependences of perceptualand affective responses are to be understood. In fact, reductive methods have been employed by many sensory
studies involving static (non-dynamic) sensory appraisal
methods [15]; however, in terms of TD methods, most of themethods mentioned earlier [6], [7], [8], [9], [10], [12], [13],[14] do not aim at a reductionistic treatment of the timeseries data acquired by TD methods. The technique in [11] iscapable of decreasing the model dimensions by using modes
computed from the time-series responses recorded by TD
tasks, but, although it excels at discriminating between different types of food products, this method does not provide
an understanding of the relationships between the sensory
and affective responses.
In the present study, by using state-space modeling,
we establish relationships between perceptual and affective
time-series responses. State-space models include measured
variables and unmeasured or latent variables to explain the
behavior of typically mechanical and information systems.
The latent variables are called states. They are not directly
observed in TD experiments, but function as memories
of past responses and are helpful in understanding therelationships between observable quantities. Usually, thenumber of state variables is smaller than the number ofperceptual and affective responses evaluated in the TD
task; therefore, state variables can provide a reductionist representation of the entire model. Further, the state-

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015

2

(a)

(b)

Sweet

Panelist 2
Panelist 3

Sour
Sour

Sweet

Sour

TD curveʤSweetʥTD curveʤSourʥ(c)0(START)T EMPORAL - DOMINANCE (TD) METHOD

In this study, we use a TD method [1], [2], [3] to study
the time evolution of perceptual and affective responses.
The TD method enables the simultaneous measurement ofmultiple types of assessors’ subjective responses to various
stimuli. Thus far, the protocols of TD methods have been
established and validated for food and beverage stimuli. TD
methods have also been tested with tactile stimuli [19] andaudio-visual stimuli [4], [20], [21], but less extensively. In
future, the TD method is expected to be a standard one formultiple sensory modalities. Here, we brieﬂy introduce themethod; more details can be found in previous reports [1],[2], [3].
In the TD method, a graphical touch-panel interface
is used, as shown in Fig. 1 (a). The assessor presses thestart button upon the initiation of food intake. Various
other buttons on the touch panel may then be pressed,
their labels corresponding to possible dominant feelings
about (or perceptions of) the food while eating it. Here,the “dominant” sensation is the sensation catching attention
at that moment, not necessarily the one with the highest
intensity. The assessor selects a corresponding button eachtime the dominant feeling changes; only one button can beselected at any given moment. The stop button is pressed
when the sensations associated with food intake are nolonger predominant. A button can be selected more thanonce, and not every button needs to be selected.
The times at which buttons are selected in each trial arerecorded. As a result, we can obtain a binary time-series
for each button as shown in Fig. 1 (b). We can calculate TD
curves by integration and smoothing the binary time-series
from all trials (Fig. 1 (c)). The horizontal axis of a TD curveis the normalized time with zero at the moment of pushing
the start button and one at that of pushing the stop button.
The vertical axis is the dominance rate, the ratio between
how often the button was selected at each time and the total
number of trials (i.e., the number of assessors × the numberof trials per assessor).

Sweet

Sweet

Panelist 1

Dominance rate (%)

variable model allows us to estimate the affective responses
as outputs when the perceptual responses are provided as
inputs. None of the previous approaches [6], [7], [8], [9],[10], [11], [12], [13] can simultaneously offer a reductionist
representation and the prediction of affective responses.
The present study is based on [16], in which we introduced the state-space modeling of TD responses to strawberries. In this study, we establish state-space models of TD
responses to strawberries from 10 participants [6], [7] andplum pickles from 20 participants [11], [17]. The model parameters are determined by cross-validation approach suchthat the observation and estimation of TD responses exhibit
a high correlation coefﬁcient, which was not performedin [16]. For this purpose, a number of samples are attainedby a bootstrap sampling method. We discuss the semantic
validity of the model (chieﬂy in the case of strawberries, asplum pickles are popular in Japan but possibly not popularin much of the world). We note that the method developed
here is also applicable to data recorded by temporal checkall-that-apply method [18], a popular modiﬁcation of a TD
method.

2

Normalized time

1
(STOP)

Fig. 1. Overview of temporal-dominance (TD) methods. (a) Example
of a graphical user interface used in TD experiments. (b) Binary dataobtained by TD method from three assessors (panelists). (c) TD curves
calculated by accumulating and smoothing the panelists’ binary responses. Adapted from [16].

TD methods are used for recording perceptual responses(TD of sensations) and affective and preferential responses(TD of emotions). In general, perceptual and affective responses are measured separately in repeated trials, not in asingle trial; the data collection for the present study was also
conducted in this standard way.

3

S TATE - SPACEMODELINGOFTEMPORAL -DOMINANCE CURVES

3.1

State and observation equations

In this study, we construct a model that estimates changes
in affective response from changes in perceptual response.
The output and input vectors for the model are the discrete
differential series of affective and perceptual dominance
rates:
yt = [y1t , . . . , yit , . . . , ypt ]T ,(1)

ut = [u1t , . . . , ujt , . . . , uqt ]T ,(2)

where yit and ujt are the differential values between two
successive instants t and t + ∆t of responses to the i-th
affective and j -th perceptual descriptor, respectively. Notethat i runs from 1 to p and j from 1 to q .
The state equation and observation equation are

mt+∆t = Amt + But ,(3)

yt = Cmt + Dut + et ,(4)

where A, B, C and D are coefﬁcient matrices for eachvariable; et ∈ Rp×1 is the observation error vector, with

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015

3

mean 0; and mt ∈ Rd×1 is the vector of state variables
at time t. d is the number of state variables and can bearbitrarily determined. From (3), the present state variables
can be determined using the past state and input vectors; thepresent state variables implicitly contain information about
the past states of the system. Hence, mt is called the vectorof memory. We show a concrete method of calculating it in
Section 3.2.

3.2

Canonical variate analysis

To determine the values of the state (memory) variables
mt at instant t, we use canonical variate analysis fortime-evolving inputs and outputs [22]. The past vector′′pt ∈ Rp ×1 and future vector ft ∈ Rf ×1 are deﬁned asTTTT
pt = [yt−∆t, . . . , yt−l∆t, uTt−∆t , . . . , ut−l∆t ]

(5)

T
T
ft = [ytT , yt+∆t, . . . , yt+h∆t
]T ,

(6)

where l ∈ {1, 2, ...} and h ∈ {0, 1, 2, ...} represent time lags
(or forward steps) for the past and future, respectively. The
sizes of the vectors are p′ = l(p + q) and f ′ = p(h + 1). pt
includes the system’s past inputs and outputs at t, whereas
ft includes the current and future outputs at t.
The values of the vector of state variables mt at t arecomputed using the past vector pt :−1

mt = W T Σpp2 pt ,(7)′

where W ∈ Rp ×d is a matrix of the left singular vector
obtained by singular value decomposition, as in (8) below;′′
and Σpp ∈ Rp ×p is the variance matrix of pt . We also′′deﬁne Σf f ∈ Rf ×f as the variance matrix of ft , and′′Σpf ∈ Rp ×f as the covariance matrix of pt and ft :−1

−1

Σpp2 Σpf Σf f2 = W ΣV T .

(8)′Here, Σ ∈ Rd×d and V ∈ Rf ×d are a diagonal matrix of thesingular values and the matrix of the right singular vector,
respectively.

3.3 Bootstrap resampling
In canonical variate analysis, overﬁtting occurs unless the
sample size, i.e., the number of curve sets, is sufﬁciently
large compared to the number of variates to be analyzed.
In the TD method, a single set of TD curves is acquiredby superimposing the responses of all assessors (Fig. 1 (c)).
Hence, canonical variate analysis cannot be directly applied.
To circumvent this problem, we increase the number of TD
curves by bootstrap resampling [23], i.e., forming a new
sample set by sampling the original observational data withreplacement [11]. In this study, we generated a single setof TD curves by sampling n times at random from a poolof n participants. By repeating this process 40 times, we
produced 40 sets of TD curves, to which canonical variate
analysis was applied.

3.4

Selection of model parameters

To construct a state-space model, we need to choose thepast (l) and future (h) time lags and the number of statevariables (d). In earlier studies [24], [25], the time lags were
determined such that the correlation coefﬁcients between
observed and estimated values were maximized. However,that approach led to overﬁtting when applied to our problems. To determine the number of state variables, Bartlett’s
test or Akaike’s information criterion is typically used.
However, Bartlett’s test led to overly complicated models
for our problems (d = 9 in the case of strawberries, forexample). On the other hand, Akaike’s information criterion
led to overly simpliﬁed models (d = 1 in the case of strawberries) that were not capable of predicting the affective
time-series.
Thus, we determined the model parameters by using
two-fold cross-validation. We divided the forty sets of TD
curves generated by the bootstrap resampling into twogroups at random. We then computed the state-space modelusing the data of one group and obtained the state values(mt ) and model coefﬁcients (A–D ) corresponding to them.
The correlation coefﬁcients between the observed and estimated values were then computed using the data of theother group. The two groups were switched, and the sameprocess was repeated. The correlation coefﬁcients acquiredin this way were averaged. These procedures were conducted exhaustively for all the combination of parametersdeﬁned by l = 1, ..., 5, h = 0, ..., 10, and d = 1, ..., 6. (Thisparameter space was determined through a preliminary
search with a small number of bootstrapped samples.) We
then employed the parameter sets that achieved the greatest
correlation coefﬁcients.
3.5 Edge reduction of the model based on conﬁdence
interval
The coefﬁcients A–D were computed between all variables,
but some of them were not statistically signiﬁcant. We
used the sets of bootstrap samples, from each of which the
coefﬁcient values, along with their means and conﬁdence
intervals, were computed. If the mean of a certain coefﬁcientdiffered from zero by at least one conﬁdence interval, thenthat coefﬁcient was considered signiﬁcant and retained in
the model; insigniﬁcant coefﬁcients were dropped. This process of edge reduction greatly facilitates the interpretation of
the state-space models.

4

TD RESPONSES FOR STRAWBERRIES AND PLUMPICKLES

4.1

TD curves for strawberries

Fig. 2 shows the TD curves obtained in [6], [7] as theassessors (eight male and two female university students,
22–24 years old: n = 10) were eating strawberries. All theassessors conducted TD of both sensations and emotions
tasks. The researchers recorded the responses using a setof adjectives chosen in advance. We removed melty, soft,
happy/satisﬁed, natural, and elegant, which we judged to
be statistically meaningless according to the criteria givenin [1], and used the remaining nine descriptors (listed inTable 1) in our later analysis. The curves were smoothed

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015

4

TABLE 1
Descriptors used to represent perceptual and affective/evaluative
responses to strawberries [7].
Perceptual
Sweet
Sour
WateryRefreshing
Juicy

TABLE 2
Descriptors used to represent perceptual and affective/evaluative
responses to plum pickles [11].

Affective/Evaluative
Like
Delicious
Fresh
Flavorsome

Perceptual
Sour
Sweet
Umami
Salty
SmoothJuicy
WateryAffective/Evaluative
Like
Dislike
Delicious
Flavorsome
Rich /Deep
Sharp
Arousing

1

Sweet

Juicy

0.8

WateryRefreshing

Sour

(a)
1
Dominance rate

Dominance rate

(a)

0.6
0.4
0.2
0

0.6

Delicious

Flavorsome

Fresh

0.4
0.2
0
0.5
Normalized time

Sour

1

0.5

0

Like
Delicious

0.8

Rich/Deep
Sharp
Arousing

Dislike

0.6

Flavorsome
0.4
0.2

1

0
0

Fig. 2. Temporal-dominance (TD) curves for (a) sensory adjectives and(b) affective/evaluative adjectives for strawberries (modiﬁed from [7]).

by a low pass ﬁlter with a cutoff frequency of 1/∆t, where∆t was 1/30 the normalized time, or approximately 1 s.
Continuous TD curves were discretized into 31 points witha sampling interval of ∆t.
4.2 TD curves for plum pickles
Another published study [11], involving twenty university
student assessors (16 males and four females, 21–26 years
old: n = 20), contains TD curves for ten types of plum
pickles. Similar to the case of strawberries, all the assessors
conducted TD of both sensations and emotions tasks. We
used the curves for the bonito-ﬂavored plum pickle, shownin Fig. 3, eliminating the adjectives expensive, refreshing,
and fruity because of their small dominance rates; the 14
adjectives we retained are listed in Table 2. The smoothing
and discretization methods employed with the plum-pickle
data were the same as those with the strawberry data.

5

Watery

1

0.6

0

Umami

(b)

Dominance rate

Dominance rate

0.8

Like

Sweet

Juicy
Salty

0.4

0

(b)
1

Smooth

0.2

1

0.5

0

0.8

0.5
Normalized time

Fig. 3. Temporal-dominance (TD) curves for (a) sensory adjectives and(b) affective/evaluative adjectives for plum pickles (modiﬁed from [11].)

Table 3. The values of l and d vary among affective descriptors, whereas h = 1 is selected for all of them. The parametervalues that maximize the mean estimation accuracy for all
types of descriptors are (l, h, d) = (2, 1, 3); we used thisparameter set to construct the model for strawberries.
5.1.2 State-space model for strawberries
The state-space model for strawberries constructed using
the above parameters is shown in Fig. 4. The model is shownin a decomposed manner because it includes many edges.
TABLE 3
Parameters to maximize the estimation accuracy of individual affective
descriptors for strawberries. l: time lag for the past, h: time lag (orforward step) for the future, d: number of state variables. The
parameters in the bottom row are used for the ﬁnal model.

R ESULTS

5.1 Strawberries
5.1.1 Selected model parameters
For strawberries, the selected model parameters l (past lags),
h (future lags), and d (number of state variables) are listed in

1

Like
Delicious
Fresh
Flavorsome
Whole model

l
3
1
5
2
2

h
1
1
1
1
1

d
5
2
3
5
3

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015

In these ﬁgures, the nodes indicate perceptual responses,
affective responses, or state variables. The edges indicate
signiﬁcant inﬂuence between nodes. The values next to theedges are the coefﬁcients corresponding to each edge. Dotted edges are statistically signiﬁcant; however, their effects
are comparatively weak, with the absolute values of thecoefﬁcients less than one third that of the largest inﬂuenceon each node. Red and blue edges indicate positive andnegative inﬂuences, respectively. As deﬁned in (3) and (4),edges connected to state variables are the effects of pastvalues, and edges connected to affective adjectives are theeffects of current values.
Fig. 4 (a) shows that the ﬁrst state variable (m1 ) ismainly affected by juicy and sweet and exerts a powerful
effect on delicious. Therefore, it represents the memory ofthe deliciousness of strawberries.
Fig. 4 (b) shows that the second state variable (m2 )is strongly affected by watery; inﬂuences from the otherperceptual responses are small. This state variable represents the memory of wateriness of strawberries; it exhibits
negative effects on like and fresh and positive effects on
delicious and ﬂavorsome.
The third state variable (m3 ), shown in Fig. 4 (c), isnegatively affected by watery and positively affected byrefreshing and sour. This state variable may represent the
memory of the coolness or unripeness of strawberries. This
state exhibits a positive effect on like and a negative effecton ﬂavorsome.
Figs. 4 (d)–(g) show the parts of the model pertaining
to each affective word. The node like is inﬂuenced by thememories of wateriness (m2 ) and coolness (m3 ) and by
the perceptual variables watery, refreshing, and sour. The
node delicious is mainly inﬂuenced by refreshing and by
the memories of deliciousness (m1 ) and wateriness (m2 ).
The node fresh is weakly inﬂuenced by several perceptual
responses and memories; therefore, it must be considered
a comprehensive subjective response. The node ﬂavorsome
is mainly inﬂuenced by refreshing and m1 (memory of deliciousness), but is also moderately affected by the other two
types of memories, i.e., wateriness (m2 ) and coolness (m3 ).
The semantic validity of these connections will be discussed
in Section 6.1.
5.1.3 Estimation accuracy for strawberries
Fig. 5 shows the observed and estimated TD curves for affective descriptors (in orange and blue, respectively). These
curves are the means of resampled TD curves. Note thatthe curves are not shown at very small normalized times
because the past values for the perceptual descriptors andstates are not deﬁned for t < 0 and the estimate for t < l∆tis also not deﬁned. The correlation coefﬁcients between
the observed and estimated values are listed in Table 4.
These were computed from the 29 discrete points in Fig. 5,
excluding the ﬁrst two undeﬁned points. The correlation
coefﬁcients for delicious and for ﬂavorsome are approximately
0.90, and thus they are predicted with sufﬁcient accuracy.
By contrast, like and fresh are only moderately correlated. In
terms of fresh, despite its moderate correlation coefﬁcient,i.e., 0.49, the estimated and observed curves are substantially different, suggesting a difﬁculty in estimating freshusing the sensory responses (see also Section 6.1).

5

TABLE 4
Correlation coefﬁcients between observed and estimatedtemporal-dominance curves for strawberries
Adjective
Like
Delicious
Fresh
Flavorsome

Correlation coefﬁcient
0.54
0.88
0.49
0.94

TABLE 5
Parameters to maximize estimation accuracy of individual affective
descriptors for plum pickles. l: time lag for the past, h: time lag (orforward step) for the future, d: number of state variables. The
parameters in the bottom row are used for the ﬁnal model.

Like
Delicious
Dislike
Flavorsome
Rich and deep
Sharp
Arousing
Whole

5.2
5.2.1

l
1
4
4
4
1
2
2
2

h
5
5
5
3
1
1
5
4

d
3
1
1
1
4
2
1
2

Plum pickles
Selected model parameters

The selected model parameters for plum pickles are listed
in Table 5. Like those for strawberries, they vary among
affective descriptors. The parameter set (l, h, d) = (2, 4, 2) isbest in terms of the average correlation coefﬁcients among
all types of affective descriptors, and we used it for the
analysis of plum-pickle data.
5.2.2

Computed state-space models for plum pickles

The state-space models related to state variables for plum
pickles are shown in Fig. 6. The models speciﬁc to affective
adjectives are shown in the Appendix.
Fig. 6 (a) is the model related to the ﬁrst state variable.
This state is positively affected by the second state variable
and by all perceptual responses except sweet. It is especiallyaffected by salty, sour, and umami, which are the generaltastes of bonito-ﬂavored plum pickles [11]. Also, the ﬁrst
state variable exhibits strong effects on like and delicious. This
state may correspond to the overall reaction to plums, or thevalence.
Fig. 6 (b) is the model related to the second state variable.
This state is strongly affected by sweet and juicy, from which
we infer that it may represent the memory of the sweetnessof fruit.
5.2.3

Estimation accuracy for plum pickles

Fig. 7 shows the observed and estimated TD curves for plum
pickles. Their correlation coefﬁcients, summarized in Table 6
were computed using the 29 discrete points, excluding theﬁrst two points. The correlation coefﬁcients for delicious,
dislike and rich and deep are equal to or greater than 0.8,
suggesting that the model represents them accurately. By
contrast, the correlation coefﬁcients for the other attributes
are moderate or low, with that for sharp being the smallest
at 0.36.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015

6
Like

Like

Fresh

Delicious
0.54

Fresh

Delicious

Flavorsome

Flavorsome

0.34

െ0.32

0.29

െ0.07

݉ଵ

݉ଶ

݉ଷ

0.22
0.40
݉ଵ

݉ଶ

݉ଷ
0.10

0.52

െ0.10

0.18

െ0.14

0.19

0.39

െ0.52

0.59
Watery

Juicy

Sour

Refreshing

Sweet(d) State-space model related to Like.
Juicy

WateryRefreshing

Sour

Sweet

Fresh

Delicious

Like

Flavorsome(a) State-space model related to the first state variable: memory ofdeliciousness.
0.32

0.54

݉ଵ

Like

Fresh

Delicious

Flavorsome

െ0.08
0.16

0.32

െ0.32

0.34

0.38

Juicy

݉ଶ

݉ଷ

0.72

Watery

Sour

Sweet

Fresh

Flavorsome

0.15

െ0.17

WateryRefreshing

Delicious

െ0.07

Juicy

0.14

െ0.11

(e) State-space model related to Delicious.
Like

െ0.13

െ0.23

0.12

െ0.36

݉ଵ

݉ଷ

݉ଶ

Refreshing

Sweet

Sour

݉ଵ

െ0.36

െ0.19

݉ଶ

݉ଷ

(b) State-space model related to the second state variable: memoryof wateriness.

0.14

0.28

0.31

0.26
0.13

Fresh

Delicious

Like

Flavorsome

Juicy

WateryRefreshing

Sour

Sweet

(f) State-space model related to Fresh.
0.34

െ0.19

െ0.12
Fresh

Delicious

Like

Flavorsome

െ0.10

െ0.06
െ0.18
݉ଵ

݉ଶ

݉ଷ
0.29

െ0.36

Juicy

Watery0.98

Refreshing

0.37

Sour

݉ଵ

0.31

0.12

െ0.12

݉ଶ

݉ଷ

Sweet

(c) State-space model related to the third state variable: memory of
coolness or unripeness.

െ0.36

Juicy

WateryRefreshing

െ0.07

Sour

Sweet

(g) State-space model related to Flavorsome.
Fig. 4. State-space models for strawberries related to (a) the ﬁrst state variable (memory of deliciousness), (b) the second state variable (memoryof wateriness), (c) the third state variable (memory of coolness or unripeness), (d) like, (e) delicious, (f) fresh, and (g) ﬂavorsome.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015

Dominance rate

0.5(a) Observed and estimated TD curves for Like
R = 0.54 (27  3.33,  0.0025)

Like

Delicious

0.21

0.25

Dislike Flavorsome

Rich / Deep

0.11

െ0.10

െ0.08

െ0.12

Sharp

Arousing

0.30

0

0.5

Dominance rate

Estimated
Observed

0.070(b) Observed and estimated TD curves for Delicious
R = 0.88 ( 27  9.62,  3.18  10ିଵ଴ )

݉ଶ

݉ଵ

1

Normalized time

Estimated
Observed
0.53

0.20

0.37

0.40

0.55

0.79

0
0

0.5

Dominance rate

71Normalized time(c) Observed and estimated TD curves for Fresh
R = 0.49 (27  2.92,  0.0070)

Sour

Sweet

SmoothJuicy

Watery

Umami

Salty

(a) State-space model related to the first state variable: memory of
average experience.

Estimated
Observed

Like

Dislike

Delicious

Flavorsome
0.05

െ0.05

െ0.11

െ0.14

Rich / Deep
0.10

Sharp
0.06

Arousing
0.12

0
0

1
Normalized time
െ0.40

݉ଵ
Dominance rate(d) Observed and estimated TD curves for Flavorsome
0.5
R = 0.94 ( 27  14.3,  3.95  10ିଵସ )Estimated
Observed

0.37

0
0

Normalized time

Fig. 5. Observed and estimated temporal-dominance (TD) curves forstrawberries: (a) like, (b) delicious, (c) fresh, and (d) ﬂavorsome. R is thecorrelation coefﬁcient between the observed and estimated values in the
curves. Results of the signiﬁcance tests of correlation are also shown.
TABLE 6
Correlation coefﬁcients between observed and estimatedtemporal-dominance curves for plum pickles
Correlation coefﬁcients
0.51
0.96
0.85
0.44
0.80
0.36
0.48

6

D ISCUSSION

6.1

Semantic validity of the model for strawberries

2.10

0.93

െ0.43

0.33

1

Sour

Adjective
Like
Delicious
Dislike
Flavorsome
Rich and deep
Sharp
Arousing݉ଶLike that of previous studies [7], [9], [10], [11], [12], [13], themodeling method in this one is meant to provide insight
into the dynamics of perceptual and affective responses. It
is therefore important to consider whether the interpretation
put on the model is semantically valid. We discuss this
only in the case of strawberries by citing some literature,
because, as mentioned previously, plum pickles are eaten
in very limited regions of the world or in certain cultures,and the validity of the model may not be evident to readers

Smooth

Sweet

Juicy

Watery

Umami

Salty

(b) State-space model related to the second state variable: memoryof sweetness of fruit.
Fig. 6. State-space models for plum pickles related to (a) the ﬁrst statevariable: memory of overall reaction; and (b) the second state variable:
memory of sweetness of fruit.

unfamiliar with this food.
The three types of states that could be considered as predominant memories were found in the strawberry model.
These memories correspond well to the principal components found by Oliver et al. [26] in their study where several
types of strawberries were classiﬁed on the basis of a ratingtask using normative descriptors, i.e., quantitative descriptive analysis. According to [26], the positive extreme of theﬁrst principal component was characterized by the sweetand ﬂoral ﬂavor and fruity aroma. These characteristics areclose to those of the ﬁrst memory (memory of deliciousness)in our model. The negative extreme of the ﬁrst principal
component was characterized by the under-ripeness suchas green ﬂavor and aroma, sourness, and citrus ﬂavor andaroma. These characteristics are close to those of the third
memory (memory of coolness and unripeness) in our model.
The second principal component in [26] was mostly related
to the appearances of strawberries, which were not investigated in the present study. The third principal componentin [26] was characterized by the quantity of ﬂuid in strawberries, which corresponds to the substances of our second

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015

Dominance rate

0.5(a) Observed and estimated TD curves for Like
R = 0.51 ( 27  3.08,  0.0047)

Dominance rate

0.5

Dominance rate1

Normalized time(c) Observed and estimated TD curves for Dislike
R = 0.85 ( 27  8.38,  5.38 10ିଽ )Estimated
Observed

00

0.5

Dominance rate

Estimated
Observed

0

0.5

1

Normalized time(d) Observed and estimated TD curves for Flavorsome
R = 0.44 ( 27  2.54,  0.0169)

Estimated
Observed

00

0.5

Dominance rate1

Normalized time(b) Observed and estimated TD curves for Delicious
R = 0.96 ( 27  17.8,  2.22 10ିଵ଺ )01

Normalized time(e) Observed and estimated TD curves for Rich/Deep
R = 0.80 ( 27  6.93,  1.91 10ି଻ )Estimated
Observed

00

0.5

Dominance rate

Estimated
Observed

001

Normalized time(f) Observed and estimated TD curves for Sharp
R = 0.36 ( 27  2.01,  0.055)

Estimated
Observed

00

0.5

Dominance rate

81Normalized time(g) Observed and estimated TD curves for Arousing
R = 0.48 ( 27  2.84,  0.0084)

Estimated
Observed

00Normalized time1

Fig. 7. Observed and estimated temporal-dominance (TD) curves forplum pickles: (a) like, (b) delicious, (c) dislike, (d) ﬂavorsome, (e) richand deep, (f) sharp, and (g) arousing. Results of the signiﬁcance tests ofcorrelation are also shown.

memory (memory of wateriness). The study by Oliver et
al. [26] did not investigate the dynamic sensory appraisal;
however, some common features can be found in [26] andour dynamic sensory appraisal.
As shown in Fig. 4 (e), delicious was positively inﬂuenced
by the memories of deliciousness (m1 ) and wateriness (m2 ).
The memory of deliciousness is an accumulation of pastjuicy and sweet. Hence, this part of the model is interpreted
as stating that sweeter, juicier, and waterier strawberries are
judged more delicious. This appears plausible for strawberries.
As shown in Fig. 4 (d), like was negatively affected bywatery and the memory of wateriness (m2 ). It is difﬁcult tosay whether or not this connection is reasonable; however, ina sensory study of strawberries [27], the amount of liquid instrawberries was neutral or negative against the consumer
preferences, which is consistent with our results. By contrast, refreshing and sour positively inﬂuenced like, as did thememory of coolness or unripeness (m3 ). Thus, strawberries
that felt cool, refreshing, and sour were preferred. For mostfruits, sweetness is a positive factor [27], [28], [29], [30];however, in our data, sweet hardly affected like. Instead, as
mentioned above, sweet pertained to delicious. In most of theearlier studies, either like or delicious was used in ratings,
but not both; the meanings of these two descriptors might
have been mixed. One might expect that the responses todelicious and like would be similar; however, in our data,
they were inﬂuenced differently by watery and refreshing.
This dissociation between deliciousness and liking was seenin our past analysis using the same data [7]. Ares et al.
also reported that deliciousness and liking responses weredifferent in their study using milk desserts [31], but thatdeliciousness can be a driver of liking.
In terms of fresh, our model may be inadequate, given the
low correlation coefﬁcient (0.49) between observation andestimation. As shown in Fig. 4 (f), fresh received positive effects from all the perceptual responses, and negative effects
from the memories of wateriness (m2 ) and coolness (m3 ).
Although the direct effects of watery, refreshing, and sour on
fresh were positive, the indirect effects through m2 and m3
were negative. It is difﬁcult to explain these connections
semantically. We speculate that the perceptual descriptors
used in our data did not cover the information necessary
to judge the freshness of strawberries. According to Péneau
et al. [32], although the subjective judgment of freshness
in strawberries and apples is inﬂuenced by many factors,
including odor and texture, visible aspects such as bruises
and surface glossiness are the most important. These were
not addressed in the questionnaire that we used.
Flavorsome represents the good odor. It was positively
affected by the memories of deliciousness (m1 ) and wateriness (m2 ). It is reasonable that the memory of deliciousness
would positively inﬂuence ﬂavorsome, because this memory
is composed of the time-lagged descriptors juicy and sweet.
As shown in Fig. 4 (e) and (g), ﬂavorsome and delicious have
similar structures. This seems fair, because taste preference
for strawberries is strongly related to their aroma [30], [33],[34].
In general, our model of strawberries seems to be semantically acceptable. At the very least, it does not include
apparently incorrect connections between the descriptors.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015

Nonetheless, this point should be further studied involving
various type of foods and stimuli in future.
Here, the strawberry model of the present study andthat of our past report [16] are compared. In [16], themodel parameters were arbitrarily determined ((l, h, d) =(1, 1, 3)) whereas in the present study, they were determinedsuch that the mean prediction performance was maximized((l, h, d) = (2, 1, 3)) as in Section 3.4. In terms of delicious
and ﬂavorsome, the correlation coefﬁcients between the observation and estimation are similar between the present
model and that in [16]. The correlation coefﬁcient for likein the present model decreased to 0.54 from 0.81 in [16].
In contrast, the coefﬁcient for fresh increased to 0.49 from0.09. The mean correlation coefﬁcient of the present modelis 0.71, which is slightly greater than 0.66 in [16]. Thisis understandable because the parameters in the present
model were selected to maximize the mean correlation
coefﬁcient. Nonetheless, a criterion for parameter selection
should be adjusted according to the objective of the model
establishment.
6.2

Limitations of the method and study

We constructed models to estimate the changes in affective
responses from those in perceptual responses. In reality,
affective and perceptual responses may exert a mutualinﬂuence, and their effects may not be unilateral. Further,affective responses are known to inﬂuence each other [35],[36], [37]. Our method does not incorporate such factors,
perhaps limiting the realism of the model. Perceptual andaffective responses can inﬂuence each other in the model
presented in [7]: a model, however, that does not allow
the introduction of latent memory variables. Multi-layered
modeling methods [37], [38], [39], on the other hand, may
mitigate the response-interaction problem more effectively:
such models consider multiple layers of affective responses
whereas the present model has only a ﬂat output layer.
As may be seen from Tables 3 and 5, time-lag parameters
differ among the descriptors. This is interesting and should
be studied in future. However, our method uses the sametime lag for all descriptors chosen so that the average abilityof the model is maximized. In future, model parameter
ﬂexibly determined by descriptors should be substituted forthe present model’s unique time-lag values.
One concern about the present study is that the bootstrap
resampling method for TD data has not been validated.
Thus far, the same or similar approaches have rarely been
employed and its adverse effects such as the biases of dominance rates are unknown. However, samples made by theresampling method do not include impossible values, suchas negative dominance rates and sums of rates exceeding
100% at any instances.
There is no established requirement in the food industry
or among researchers for the minimum estimation-accuracyof TD curves. Okada et al. [7] estimated the TD curves ofstrawberries using the same data as the present study andvector auto-regression modeling. Compared to their crossvalidation results, our model appears better: the correlation
coefﬁcients between the observed and estimated TD curvesof the present model are higher than those in [7] by 0.1–
0.45 points. However, this difference may simply be due

9to the higher number of connections among variables andthe greater complexity of the present model. The criteria
of model selection differ between modeling methods, and asingle performance value is an inadequate basis for comparison. In future, methods for performance comparison should
be developed.

7

C ONCLUSION

We proposed a state-space modeling method for temporaldominance responses and tested it on TD data for strawberries and plum pickles. We computed the state variables
using canonical correlation analysis, which was possible
once we had prepared a large set of samples by bootstrap resampling. We determined the model parameters bycross-validation. The model included memory variables thatsummarized the directly observed variables reductively. For
example, for strawberries, the dynamic changes of nine
types of TD curves were connected by three types of factorial memories: memories of deliciousness, coolness, andwateriness. These results suggest that these three memories play prominent roles in the time-dependent subjective
experience of eating strawberries. The semantic validity ofthe model was assessed and found to be acceptable. The
model partly predicted the TD curves of affective responses;
however, some aspects, including the ﬂexible determinationof model parameters, should be explored further to improvethe model’s utility.

A PPENDIX AF IGURES OF THE STATE - SPACE MODEL FOR PLUMPICKLES

Fig. 8 shows the state-space model for plum pickles. Eachof Fig. 8 (a)–(g) features the inﬂuences on an individual
affective descriptor.

ACKNOWLEDGMENTS
This study was in part supported by MEXT Kakenhi(15H05923).

R EFERENCES
[1]

[2]
[3]
[4]

[5]
[6]

N. Pineau, P. Schlich, S. Cordelle, C. Mathonnière, S. Issanchou,
A. Imbert, M. Rogeaux, P. Etiévant, and E. Köster, “Temporal dominance of sensations: Construction of the TDS curves and comparison with time–intensity,” Food Quality and Preference, vol. 20, no. 6,
pp. 450–455, 2009.
R. Di Monaco, C. Su, P. Masi, and S. Cavella, “Temporal dominance of sensations: A review,” Trends in Food Science & Technology,
vol. 38, no. 2, pp. 104–112, 2014.
P. Schlich, “Temporal dominance of sensations (TDS): a new deal
for temporal sensory analysis,” Current Opinion in Food Science,
vol. 15, pp. 38–42, 2017.
K. Kantono, N. Hamid, D. Shepherd, Y. H. T. Lin, S. Skiredj, andB. T. Carrd, “Emotional and electrophysiological measures correlate to ﬂavour perception in the presence of music,” Physiology&amp; Behavior, vol. 199, pp. 154–164, 2019.
D. Labbe, P. Schlich, N. Pineau, F. Gilbert, and N. Martin, “Temporal dominance of sensations and sensory proﬁling: A comparative
study,” Food Quality and Preference, vol. 20, no. 3, pp. 216–221, 2009.
T. Okada, S. Okamoto, Y. Yamada, and T. Ishikawa, “Vector autoregression model of temporal perceptual and affective responses
towards food,” in Proceedings in IEEE Global Conference on Life
Sciences and Technologies, 2019, pp. 43–45.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015
Like

Dislike

Delicious

Flavorsome

0.21

Sharp

Rich/Deep10

Arousing

Like

Delicious

DislikeRich/Deep

Flavorsome

Sharp

െ0.14
0.10

െ0.12

െ0.31

Smooth

0.58

െ0.26

Sweet

Juicy

0.21

0.09

െ0.15

Watery

Umami

Salty

(a) State-space model related to Like.
Like

Delicious

Dislike

Flavorsome

Rich/Deep

Sour

െ0.18

Smooth

Sweet

Sharp

Delicious

DislikeSour

SmoothDelicious

0.39

0.25

െ0.32
Juicy

WateryFlavorsome

Umami

Juicy

Sharp

SmoothWateryRich/Deep

݉ଵ

݉ଶ

Juicy

Salty

Sharp

Arousing

݉ଶ

Salty

Sour

Smooth

Sweet

0.39

0.26

െ0.34

Juicy

0.17

Watery

Umami

Salty

Arousing

Like

Delicious

Dislike

Flavorsome

Rich/Deep

Sharp

Arousing

Umami

0.14

Salty

0.14

െ0.09

Sour

Smooth

Sweet

0.12

݉ଶ

0.16

Juicy

0.09

Watery

Umami

0.17

Salty

(g) State-space model related to Arousing.

െ0.11

Sweet

Umami

0.06

݉ଵ

0.09

0.11

0.55

Rich/Deep

Flavorsome

0.07

0.16

Flavorsome

Watery(f) State-space model related to Sharp.
Rich/Deep

0.49

െ0.21

Sour

Dislike0.30

݉ଶDislikeJuicy݉ଵ(c) State-space model related to Dislike.
Like

Sweet

0.05

Sweet

0.42 0.22

0.31

െ0.10

݉ଵ

0.18

Delicious

݉ଶ

െ0.08

0.11

Like

Arousing(b) State-space model related to Delicious.
Like

Smooth

െ0.05

݉ଵ

0.11

Sour

0.55

(e) State-space model related to Rich/deep.

0.25

െ0.16

݉ଶ

݉ଵ

݉ଶ

݉ଵ

Sour

Arousing

Sharp

Arousing

Umami

Salty

െ0.42

Watery(d) State-space model related to Flavorsome.

Fig. 8. State-space models for plum pickles related to (a) like, (b) delicious, (c) dislike, (d) ﬂavorsome, (e) rich and deep, (f) sharp, and (f) arousing.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015

[7]

T. Okada, S. Okamoto, and Y. Yamada, “Affective dynamics:
Causality modeling of temporally evolving perceptual and affective responses,” IEEE Transactions on Affective Computing, 2019,
DOI: 10.1109/TAFFC.2019.2942931.
[8] B. Franczak, R. Browne, P. McNicholas, J. Castura, and C. Findlay,“A Markov model for temporal dominance of sensations (TDS)
data,” in Proceedings of 11th Pangborn Sensory Science Symposium,
2015.
[9] G. Lecuelle, M. Visalli, H. Cardot, and P. Schlich, “Modelingtemporal dominance of sensations with semi-Markov chains,”
Food Quality and Preference, vol. 67, pp. 59–66, 2018.
[10] H. Cardot, G. Lecuelle, P. Schlich, and M. Visalli, “Estimating ﬁnite
mixtures of semi-markov chains: an application to the segmentation of temporal sensory data,” Journal of the Royal StatisticalSociety: Series C (Applied Statistics), vol. 68, no. 5, pp. 1281–1303,
2019.
[11] S. Okamoto, Y. Ehara, T. Okada, and Y. Yamada, “Affective dynamics: Principal motion analysis of temporal dominance of sensations
and emotions data,” IEEE Transactions on Affective Computing, 2020,
DOI: 10.1109/TAFFC.2020.2971700.
[12] J. C. Castura, “Investigating temporal sensory data via a graphtheoretic approach,” Food Quality and Preference, vol. 79, p. 103787,
2020.
[13] S. Okamoto, “Structural modeling of temporal dominance responses using covariances of contemporary changes in subjective
qualities,” International Journal of Affective Enginering, vol. 20, no. 3,
pp. 126–129, 2021.
[14] M. Visalli, B. Mahieu, A. Thomas, and P. Schlich, “Concurrent vs.
retrospective temporal data collection: Attack-evolution-ﬁnish as asimpliﬁcation of temporal dominance of sensations?” Food Quality
and Preference, vol. 85, p. 103956, 2020.
[15] C. Peltier, M. Visalli, and P. Schlich, “Comparison of canonicalvariate analysis and principal component analysis on 422 descriptive sensory studies,” Food Quality and Preference, vol. 40, pp. 326–
333, 2015.
[16] K. Tachi, S. Okamoto, Y. Akiyama, and Y. Yamada, “State-spacemodeling of temporal dominance data: A case study using strawberry,” in Proceedings of the 8th International Conference on Kansei
Engineering and Emotion Research, vol. 1256, 2020, pp. 139–148.
[17] Y. Ehara, S. Okamoto, T. Okada, and Y. Yamada, “Comparison of
the method of temporal dominance of sensations and after-tasting
sensory evaluation,” in IEEE Global Conference on Life Sciences andTechnologies, 2019, pp. 210–211.
[18] J. C. Castura, L. Antúnez, A. Giménez, and G. Ares, “Temporalcheck-all-that-apply (tcata): A novel dynamic method for characterizing products,” Food Quality and Preference, vol. 47, pp. 79–90,
2016.
[19] L. Boinbaser, M. E. Parente, J. C. Castura, and G. Ares, “Dynamic
sensory characterization of cosmetic creams during application
using temporal check-all-that-apply (TCATA) questions,” Food
Quality and Preference, vol. 45, pp. 33–40, 2015.
[20] C. Peltier, M. Visalli, and A. Thomas, “Using temporal dominanceof emotions at home. impact of coffee advertisements on consumers’ behavior and methodological perspectives,” Food Quality
and Preference, vol. 71, pp. 311–319, 2019.
[21] T. C. Merlo, I. Soletti, E. S. na, B. S. Menegali, M. M. Martins,
A. C. B. Teixeira, S. dos Santos Harada-Padermo, M. D. Dargelio,
and C. J. Contreras-Castillo, “Measuring dynamics of emotions
evoked by the packaging colour of hamburgers using temporal
dominance of emotions (TDE),” Food Research International, vol.
124, pp. 147–155, 2019.
[22] W. E. Larimore, “Canonical variate analysis in control and signal
processing,” in Statistical methods in control and signal processing,T. Katayama and S. Sugimoto, Eds. Marcel Dekker, 1997, pp.
83–120.
[23] B. Efron and R. J. Tibshirani, An introduction to the bootstrap. CRC
press, 1994.
[24] W. E. Larimore, “Canonical variate analysis in identiﬁcation, ﬁltering, and adaptive control,” in 29th IEEE Conference on Decisionand control. IEEE, 1990, pp. 596–604.
[25] A. Furukawa, J. Kiyono, and H. Otsuka, “Structural matrix identiﬁcation using microtremor measurements based on canonicalvariate analysis,” Journal of Applied Mechanics, vol. 8, pp. 85–93,
2005.
[26] P. Oliver, S. Cicerale, E. Pang, and R. Keast, “Developing a
strawberry lexicon to describe cultivars at two maturation stages,”
Journal of Sensory Studies, vol. 33, no. 1, p. e12312, 2018.

11[27] ——, “Check-all-that-applies as an alternative for descriptive analysis to establish ﬂavors driving liking in strawberries,” Journal of
Sensory Studies, vol. 33, no. 2, p. e12316, 2018.
[28] X. Huang and F.-H. Hsieh, “Physical properties, sensory attributes,
and consumer preference of pear fruit leather,” Journal of Food
Science, vol. 70, no. 3, pp. E177–E186, 2005.
[29] A. K. Thybo, B. F. Kühn, and H. Martens, “Explaining Danish
children’s preferences for apples using instrumental, sensory anddemographic/behavioural data,” Food Quality & Preference, vol. 15,
no. 1, pp. 53–63, 2004.
[30] R. Bhat, J. Geppert, E. Funken, and R. Stamminger, “Consumers
perceptions and preference for strawberries-a case study from
germany,” International Journal of Fruit Science, vol. 15, no. 4, pp.
405–424, 2015.
[31] G. Ares, A. Giménez, C. Barreiro, and A. Gámbaro, “Use of an
open-ended question to identify drivers of liking of milk desserts.
comparison with preference mapping techniques,” Food Quality
and Preference, vol. 21, no. 3, pp. 286–294, 2010.
[32] S. Péneau, P. B. Brockhoff, F. Escher, and J. Nuessli, “A comprehensive approach to evaluate the freshness of strawberries andcarrots,” Postharvest Biology and Technology, vol. 45, no. 1, pp. 20–
29, 2007.
[33] J. T. V. de Resende, L. K. Camargo, E. J. Argandoña, A. Marchese,
and C. K. Camargo, “Sensory analysis and chemical characterization of strawberry fruits,” Horticultura Brasileira, vol. 26, no. 3, pp.
371–374, 2008.
[34] C. Jouquand, C. Chandler, A. Plotto, and K. Goodner, “A sensory
and chemical analysis of fresh strawberries over harvest dates andseasons reveals factors that affect eating quality,” Journal of theAmerican Society for Horticultural Science, vol. 133, no. 6, pp. 859–
867, 2008.
[35] X. Chen, C. J. Barnes, T. H. C. Childs, B. Henson, and F. Shao,“Materials’ tactile testing and characterization for consumer products’ affective packaging design,” Materials and Design, vol. 30, pp.
4299–4310, 2009.
[36] S. Okamoto, H. Kojima, A. Yamagishi, K. Kato, and A. Tamada,“Layered-modeling of affective and sensory experiences using
structural equation modeling: Touch experiences of plastic surfaces as an example,” IEEE Transactions on Affective Computing,vol. 12, no. 2, pp. 429–438, 2021.
[37] S. Okamoto, H. Nagano, K. Kidoma, and Y. Yamada, “Speciﬁcation of individuality in causal relationships among texturerelated attributes, emotions, and preferences,” International Journalof Affective Engineering, vol. 15, no. 1, pp. 11–19, 2016.
[38] I. H. M. Hashim, S. Kumamoto, K. Takemura, T. Maeno, S. Okuda,
and Y. Mori, “Tactile evaluation feedback system for multi-layered
structure inspired by human tactile perception mechanism,” Sensors, vol. 17, p. 2601, 2017.
[39] K. Kidoma, S. Okamoto, H. Nagano, and Y. Yamada, “Graphicalmodeling method of texture-related affective and perceptual responses,” International Journal of Affective Engineering, vol. 16, no. 1,
pp. 27–36, 2017.

Kyoichi Tachi is a mechanical engineer who
received a BS in engineering from Nagoya University in 2020. His research interests include
affective engineering and biomechanics.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015

Shogo Okamoto received his PhD in Information Sciences from Tohoku University in 2010.
He had been with Nagoya University since 2010;
currently, he is an associate professor at Department of Computer Sciences, Tokyo Metropolitan
University. His research interests include haptics, assistive robotics, and affective engineering.

12