ISASE 2023

Hierarchical Clustering Analysisof Temporal Dominance of Sensations TasksHiroharu NATSUME*, Shogo OKAMOTO*, and Hikaru NAGANO*** Tokyo Metropolitan University, 6-6, Asahigaoka, Hino, Tokyo, Japan** Kobe University, 1-1, Rokkodai, Nadaku, Kobe, Hyogo, Japan

Abstract: Temporal dominance of sensation (TDS) is used to obtain the time-series of multiple types of sensory responses during foodconsumption. Results of existing analysis methods for TDS are the averages of all panels. Individual differences and their time-series
similarity have seldom been studied. We defined a dissimilarity index to compare individual TDS trials and performed hierarchicalclustering on TDS data of strawberries using the index. Fifty-one trials were classified into four clusters with different characteristics.
The method will be useful for individual analyses of time-series sensory evaluation data.
Keywords: Dissimilarity, Hierarchical clustering, Strawberry

1. INTRODUCTION

interface. The computer screen displays buttons with shortadjectives (attribute words) that describe sensations suchas “sweet” and “sour.” Start/stop buttons are alsodisplayed to control the start and end of the task. In thecase of eating food, the subject (panel) puts a piece of foodin their mouth and presses the start button to begin the task.
Thereafter, at any time during tasting, the panel presses
the button with the attribute word suitable to express themost dominant sensation they feel. It is not possible toselect more than one attribute at the same time. Once abutton is pressed, it remains selected until the next button
is pressed. There may be attributes that are never selected
throughout the task, and there may be attributes that are
selected multiple times. When the food is swallowed, thepanel terminates the task by pressing the stop button.
TDS curves are typically used to visualize theThe temporal dominance of sensations (TDS) method[1] is a time-series sensory evaluation method formeasuring sensory changes over time while eating or
watching a movie. This method has been widely usedsince the mid-2010s, mainly in the field of food science,
because it allows the efficient collection of the temporal
evolution of multiple types of sensations.
Most of the TDS analysis methods developed so far
have dealt with average responses among all panels [1-4].
Only a few methods are available for multi-layer orindividual analyses. An example is the study by Cardot etal., which focused on groups of panels with similar
responses in TDS tasks [5]. The authors used a semiMarkov chain model to classify panels into several groups.
Lepage et al. proposed a method to compare the ability ofmultiple panels to discriminate foods [6]. However, nomethod has been developed to classify or compareindividual TDS trials based on their time-series similarity.
We defined a dissimilarity index for comparing TDS
trials based on their time-series similarities. The purposeof this study is to perform a hierarchical clustering of TDS
trials using this index and to examine its validity. We
applied the clustering method on open data of TDS for
strawberries [7].
2. TDS TASKS AND CURVESAs shown in Figure 1, the TDS method is performed
using a computer application with a graphical user

Figure 1: Graphical user interface used for TDS tasks1

experimental results [1]. The TDS curve is the average ofthe time-series responses over all trials and shows theproportion of trials in which each attribute is considereddominant at any given time.
3. CLUSTERING ANALYSIS
3.1 Formulation of TDS trials
The time-series response obtained in the TDS task is𝑗represented by the function of time 𝑓𝑖 (𝑡) , where 𝑗 (𝑗 =1, 2, … , 𝑛) and 𝑖 (𝑖 = 1, 2, … , 𝑞) are the serial numbers oftrials and attribute words, respectively. 𝑡 is a continuous
value that is normalized from 0 to 1 for the time from start
to end of the TDS task. The function has two values; 1
when the attribute is selected, and 0 when the attribute is
not selected.
𝑗
The summation of 𝑓𝑖 (𝑡) is 1 among all 𝑗 , due to thenature of TDS that multiple attributes cannot be selected
at the same time. However, it is 0 from the start of the task
until the first attribute is selected. The length of this period(delay) varies from trial to trial. We defined a time-series𝑗with 𝑖 = 0, i.e., 𝑓0 (𝑡), to compare trials considering the𝑗length of the delay period. 𝑓0 (𝑡) takes the value 1 during
the delay period and 0 from the time the first attribute is
selected until task completion. Consequently, 𝑖 takes
values from 0 to 𝑞 (𝑖 = 0, 1, … , 𝑞).

Figure 2: Dendrogram obtained by hierarchicalclustering. Trials with the same color belong to thesame cluster. There are four clusters.

3.3 Hierarchical clustering
We performed a hierarchical clustering analysis of TDS
trials using 𝐷𝑅 as the distance measure. 𝐷𝑅 was calculated
for all pairs of trials, i.e. {𝑓 1 , 𝑓 2 , … , 𝑓 𝑛 }2 . The pair of trials
with the smallest distance constitutes a group. The
distances between two groups, or between a group andtrial, are calculated in the same way to create a new group.
This process is repeated until there is only one group.
There are several ways to define the distance between
groups, of which Ward’s method is the most common andis the method adopted in this study. Empirically, 70% of
the maximum distance between groups (the distance
between the last two remaining groups) is defined as thecriterion for clustering. Pairs of groups with a smaller thanthe criterion are considered to belong to the same cluster.

3.2 Dissimilarity index between two TDS trials
In hierarchical clustering, the distance between twosamples is used. Therefore, we defined a measure ofdissimilarity, or distance, between two TDS trials. This is
an improvement on our previously proposed index [8].
𝑗First, discretize 𝑓𝑖 (𝑡), a function of continuous time,
into arbitrary 𝑅 intervals as follows:
𝑘+1𝑅𝑗

𝐹𝑖 [𝑘] = ∫𝑘𝑅𝑗𝑓𝑖 (𝑡) 𝑑𝑡 .

3.4 TDS data of strawberries
We used the open data of TDS tasks collected byShimaoka et al. [7]. The authors performed TDS tasks forstrawberries involving 17 university student panels.
Individual panels replicated three TDS tasks, in each ofwhich one strawberry was eaten. In total, 51 tasks wereperformed (𝑛 = 51 ). Eight types of attributes ( 𝑞 = 8 )were selected through another user study. The eight
attributes were sweet, sour, fruity, green, watery, juicy,aromatic, and light.

(1)

The dissimilarity index 𝐷𝑅 for two TDS trials 𝑓 𝑎 and 𝑓 𝑏
in any 𝑅 is defined as follows:
𝐷𝑅(𝑓 𝑎,𝑓𝑏)

=1

𝑅−1

𝑞2

∑ √∑|𝐹𝑖𝑎 [𝑘] − 𝐹𝑖𝑏 [𝑘]| .

√2𝑅 𝑘=0(2)

𝑖=0Here, 𝐹𝑖𝑎 [𝑘] and 𝐹𝑖𝑏 [𝑘] at discrete interval 𝑘 ( 𝑘 =0, 1, … , 𝑅 − 1) are considered as coordinates in (𝑞 + 1)dimensional space. The Euclidean distance between thetwo coordinates is then computed. 𝐷𝑅 is the average of thedistances among all intervals. The overall value is divided
by √2 in order to fit the dissimilarity values within theinterval [0, 1]. The index is then 0 when the time-series
responses of the two discretized trials are in perfectagreement, and 1 when they are in perfect disagreement.

4. RESULTS
We performed a hierarchical clustering on the 51
strawberry TDS trials as described in Section 3.3. Thesetrials were classified into four clusters. The obtained
dendrogram is shown in Figure 2. The leftmost group in2

Figure 3-1: TDS curve of cluster 1

Figure 3-3: TDS curve of cluster 3

Figure 3-2: TDS curve of cluster 2

Figure 3-4: TDS curve of cluster 4
this cluster, the proportion of watery is consistently high.
In the latter half of the period, the proportion of green ishigher. Unlike the other three clusters, no particularattribute becomes more prominent in the latter half of thetrial. Since cluster 4 is the largest cluster, it is considered
to be a common strawberry eating experience among
many panels. In addition, sour becomes more prominent
in the middle and green in the latter half. This suggeststhat the cluster may have been formed by the presence ofunripe individuals in the strawberries used in tasks.
In the strawberry data we used, the food preferencesand detailed attributes of panels are unknown. Also,
although all the strawberries were of the same brand, thequality of individual strawberries naturally vary. Despite
these limitations, the described clustering method will
provide profound analysis data when combined with thedemographic information of panels and food properties.

the tree was designated as cluster 1, and the clusters werenumbered sequentially. The TDS curves for each cluster
are shown in Figures 3-1, 3-2, 3-3, and 3-4. The attributecurves follow the legend of Figure 3-1.
5. DISCUSSION
Each cluster has different characteristics. Cluster 1contains the trials of the panels that perceived sour in thesecond half of the task. This cluster contains 18 trials.
Along with cluster 4, it contains more than one-third of alltrials.
Cluster 2 is characterized by a higher proportion ofsweet in the latter half of the task. In the other clusters,
sweet is selected in the first half. In addition, theproportion of watery is high immediately after the start of
the task.
In cluster 3, light was selected in the latter half of thetask. Light was referred as “without remaining sweet taste”[7]. In contrast to cluster 2, this cluster is considered to bea group of panels that perceived the decay of sweetness
over time.
Cluster 4 is the largest cluster containing 19 trials. In

6. CONCLUSION
We performed a hierarchical clustering of the TDS tasks
for strawberries. Fifty-one trials were divided into fourclusters with different time-series characteristics.

3

When fresh foods such as strawberries are used, theindividual differences in the foods affect the results. If theclassification is intended to be based on purely individual
differences in the panel, it would be better to use a productof constant quality, such as a processed food. Futurestudies will separate the effects of the individual
differences in panels and food products.
ACKNOWLEDGMENT
This work was in part supported by The Tojuro Iijima
Foundation for Food Science and Technology.
REFERENCES[1] International Organization for Standardization, ISO
13299: Sensory analysis—Methodology, 2016.
[2] Francine Lenfant, Chrystel Loret, Nicolas Pineau,
Christoph Hartmann, and Nathalie Martin; Perceptionof oral food breakdown. The concept of sensory
trajectory, Appetite, 52(3), pp. 659–667, 2009.
[3] Takumu Okada, Shogo Okamoto, and Yoji Yamada;
Affective dynamics: Causality modeling of temporally
evolving perceptual and affective responses, IEEETransactions on Affective Computing, 13(2), pp. 628–639, 2019.
[4] Shogo Okamoto, Yuki Ehara, Takumu Okada, and YojiYamada; Affective dynamics: Principal motion
analysis of temporal dominance of sensations data,
IEEE Transactions on Affective Computing, 13(2), pp.
871-880, 2020.
[5] Hervé Cardot, Guillaume Lecuelle, Pascal Schlich andMichel Visalli; Estimating finite mixtures of semiMarkov chains: an application to the segmentation oftemporal sensory data, Journal of the Royal StatisticalSociety Series C (Applied Statistics), 68(5), pp. 1281–1303, 2019.
[6] Melissa Lepage, Tara Neville, Andreas Rytz, Pascal
Schlich, Nathalie Martin, and Nicolas Pineau; Panel
performance for temporal dominance of sensations,
Food Quality and Preference, 38, pp. 24—29, 2014.
[7] Nanako Shimaoka, Shogo Okamoto, Yasuhiro
Akiyama, and Yoji Yamada; Linking temporal
dominance of sensations for primary-sensory andmulti-sensory attributes using canonical correlation
analysis, Foods, 11(6), article no. 781, 2022.
[8] Hiroharu Natsume, Shogo Okamoto, and Nanako
Shimaoka; Outlier analysis in time-evolving sensory
evaluation tasks, Proceedings of IEEE 4th Global
Conference on Life Sciences and Technologies, pp.
216–217, 2022.

4