Construction of customers’ emotion model in thebespoke tailoring using evaluation grid method
Kodai Obata, Masashi Sugimoto, Saki Miyai, Yoichi Yamazaki, Fan Zhang, Michiya Yamamoto and Noriko Nagata
Kwansei Gakuin University
Sanda, Hyogo, Japan
E-mail: {kodai-O, nagata}@kwansei.ac.jp

Abstract—To realize a recommendation system that combines
the benefits of human recommendation (experts in the field, e.g.,salesclerks) and computer recommendation (recommendation
algorithms, such as those in e-commerce), we constructed anemotion model that appropriately represents customers’ emotions
during a human recommendation process. As a typical scene ofa recommendation by a human, we focused on bespoke tailoringin which a customer interacts with a tailor. First, we extracted
emotions evoked in the scene using the evaluation grid method.
After that, we constructed an emotion model by determining
the extracted emotions’ positions on the core affect model. As
a result, we constructed an emotion model that appropriately
expressed the emotions evoked in the recommendation process.
In addition, we found that unique emotions such as troubled andexpected are evoked in the recommendation by a human.
Index Terms—emotion model, human-human interaction, bespoke tailoring

I. I NTRODUCTION
With the diversification of services and products, customers
must make many decisions during purchase behavior. As aresult, there is a growing demand for recommendation systems[1] that deal with large amounts of complex information
on purchase behavior. The recommendation systems we use
can be divided into two categories: a recommendation by ahuman (such as experts in the field) or a recommendation
by a computer (such as a recommendation algorithm fore-commerce). Both recommendation systems free customers
from choices with which they cannot deal by themselves.
In other words, the recommendation system helps the customers’ decision-making in the purchase process. However, a
big difference between these two recommendation systems iswhether or not there are interactions between customers andrecommenders. In a recommendation by a human, a humanmakes a recommendation to the customer and the customer
responds to the recommendation. Although a human recommendation can garner customers’ trust [2], for humans to makea recommendation to every customer is time consuming. On
the other hand, a computer recommendation uses a customers’purchase history and other people’s purchase information to
make a recommendation, and it can provide the recommendation immediately. Thus, the two recommendation methods
both have advantages and disadvantages. A combination ofthese excellent points may offer customers more satisfying
recommendations.

To realize such a recommendation system, it is important toclarify the human-human interaction and the human-computerinteraction during the recommendation. To clarify the humanhuman interaction, Honda [3] focused on nonverbal behavior
related to people’s mental states. Honda examined the relationship between customers’ nonverbal behavior and customers’
mental states to clarify the importance of nonverbal behavior.
On the other hand, Voeffray [4] clarified the human-computerinteraction by researching the automatic detection of users’emotions to realize emotion-sensitive human-computer interactions. The emotions used in these studies were common (i.e.,
it did not assume emotion difference in different domains).
However, the emotions differed depending on the domains inwhich they were evoked, and it is unclear if they reflect specificemotions in the field [5]–[7]. Therefore, this research tried
to construct an emotion model that appropriately expresses
customers’ emotions during the recommendation process.
Human recommendations involve complex interactions,such as consultations and nonverbal behavior, and are considered suitable for the extraction of various customer emotions.
Bespoke tailoring is a typical human recommendation andhas a long history [8]. In this paper, we focused on bespoke
tailoring while talking with the tailor (Fig. 1), and we aimed
to construct an emotion model on bespoke tailoring.

Fig. 1. Examples of bespoke tailoring

We used a method from a previous study [5]–[7] thatcombines the evaluation grid method [10] and the core affect[9] to construct an emotion model via the two processes.
(1) We extracted emotions evoked by bespoke tailoring using
an evaluation grid method. (2) The extracted emotions werepositioned on the core affect, which expresses emotions in twodimensions: pleasure-displeasure and activation-deactivation.
In this paper, the distribution of emotions on the core affectspace is called the emotion model.
II. E XTRACTION OF EMOTIONS EVOKED IN A SCENE OFBESPOKE SUIT TAILORING USING AN EVALUATION GRIDMETHOD

A. Method
We conducted interviews using the evaluation grid method[10] to extract customers’ emotions during bespoke tailoring.
The evaluation grid method [10] is an individual interview
method for clarifying an individual’s evaluation constructs.
The procedure is a two-step interview with preference judgment and laddering (leading to related evaluation items).
Preference judgment is used for extracting the evaluation itemsof compared objects perceived by participants. Laddering asks
for specific factors (constructs of lower levels) that constitutethe evaluation items and for abstract value judgments (constructs of higher levels) that invoke the evaluation items. Thequestions that ask for lower-level constructs are called ladderdown questions, and those that ask for higher-level constructs
are called ladder-up questions. The participants’ evaluation
constructs are realized by repeating these questions.
We asked about the scene’s positive and negative points
compared with a ready-made suit’s purchase scene as aquestion of preference judgment. After that, we asked about
the causes of the positive and negative points and about
the emotions evoked by the positive and negative points in
response to preference judgments.
Thirty-seven customers (35 males and 2 females, 18-69
years old) participated in an interview just after completing
bespoke tailoring at a suit manufacturing company’s family
bazaar (2018/9/15-17). The participants ordered at least onesuit in consultation with the salesclerk. After completing thebespoke tailoring, the participants moved to a different room.
The interviews were basically conducted on a one-to-one basis
between the interviewer and the participant, and the participant
answered questions on the positive and negative points ofbespoke tailoring compared to the scene of purchasing readymade clothes. The interview took 10 minutes for both positive
and negative points.
In the data analysis, we used E-Grid (a visual analytics
system for the evaluation grid method) [11]. Some of theobtained evaluation items had the same meaning, so thoseevaluation items were integrated and unified.
The threshold for adjusting the displayed evaluation items
was 0.15 for the positive evaluation constructs and 0.36 forthe negative evaluation constructs so that both the positive
and negative emotion words were equal in number.
B. Results and discussion
We extracted 22 emotions. The positive emotions wereattachment, willingness, happiness, good, pleasantness, feeling
good, expectation, satisfaction, confidence, proud, uplifting,interesting, and peace of mind (Fig. 2). The negative ones wereregret, anxiety, dislike troublesomeness, confusion, surprised,impatience, worry, and hesitance (Fig. 3).
Emotions and factors that evoke them are shown in Table
I. In the bespoke tailoring, many emotions were evoked bydifferent factors, and some emotions were evoked by thecommon factors.
TABLE IT HE EMOTIONS AND THE FACTORS THAT EVOKE THEM

III. E XPLICATION OF THE NATURE OF EMOTIONS EVOKEDBY SCENES OF BESPOKE SUIT TAILORING BASED ON CORE
AFFECT

A. Method
To interpret the nature of the emotions evoked by bespoke
tailoring, 22 extracted emotions were positioned on the coreaffect along pleasure-displeasure and activation-deactivation
axes.
Twenty-seven university students and graduate students (17males, 9 females, and 1 unanswered, 20-39 years old) participated in the evaluation experiment. Using Google Forms, theFig. 2. The relationship diagram of emotions and factors of the positive points (Yellows nodes indicate emotions)Fig. 3. The relationship diagram of emotions and factors of the negative points (Yellows nodes indicate emotions)participants rated to what extent they felt pleasure-displeasure
and activation-deactivation with respect to 22 kinds of emotions, with 1: displeasure - 5: pleasure and 1: deactivation - 5:
activation..
For the obtained data, the evaluation value of each emotion
was converted from 1(min) and 5 (max) to -2(min) and 2(max). After that, the average rating value of all participants
for each emotion was found. Based on each emotion’s averagerating value, we conducted cluster analysis using the Wardmethod. The result of the cluster analysis is shown in Fig. 4.
The number of clusters was set to four, just before creatingclusters with extremely few emotions.
B. Results and discussion
An emotion model constructed by determining the positionof emotions on the core affect is shown in Fig. 5.
Cluster 1 included seven emotions: uplifting, interesting,willingness, confidence, happiness, pleasantness, and surprised. Cluster 2 contained four emotions: confusion, dislike, impatience, and anxiety. Cluster 3 contained four emotions: hesitance, regret, worry, and troublesomeness. Cluster 4 included seven emotions: satisfaction, feeling good,attachment, good, expectation, proud, and peace of mind.
These clusters were characterized as: pleasure-activation,pleasure-deactivation, displeasure-activation, and displeasuredeactivation. Furthermore, among the extracted emotions, thespecific emotions in bespoke tailoring, such as worry andexpectation, did not exist in the Russell’s original emotion
circumplex [12]. Therefore, these emotions needed to beFig. 4. The dendrogram of cluster analysis

positioned in the core affect. Our emotion model succeededin expressing these emotions in the core affect. And asin previous studies [5]–[7], this study was able to extract
emotions that did not exist in Russell’s emotion model. The
reason for being able to index unknown emotions is that weadopted dimensional models of emotion, like the core affect,that express emotions in two-dimensional space rather thanThe constructed emotion model appropriately expressed thecustomers’ emotions evoked in the recommendation process,and these emotions would be useful tools for investigatinginteractions in human recommendations.
ACKNOWLEDGMENT
This work was partially supported by JST COI Grant
Number JPMJCE1314.
R EFERENCES

Fig. 5. The emotion model in the scene of bespoke suit tailoring (Boldwords indicate the representative emotions)

categorical models of emotion as represented by Ekman’s
basic emotion theory that expresses emotions in categories(Six basic emotions: anger, disgust, fear, happiness, sadness,and surprise) [13]. These results suggest that the method of
combining the evaluation grid method and the core affect canconstruct a domain-specific emotion model.
In addition, each cluster’s representative emotions wereselected to simplify the emotion model. The two emotions
close to the cluster’s center are the representative emotionsof that cluster. Emotions close to the centroid of Cluster 1
are uplifting and interesting. Emotions close to the centroidof Cluster 2 were confusion and dislike. Emotions close to
the centroid of Cluster 3 were hesitance and regret. Emotions
close to the centroid of Cluster 4 were satisfaction and feelinggood. However, dislike in Cluster 2 is a very strong negative
expression and is considered a limited emotion in the humanrecommendation scene. Therefore, anxiety, the closest emotion
to dislike, is one of the representative emotions of Cluster 2.
IV. C ONCLUSION
In this paper, we constructed an emotion model for bespoke
tailoring by extracting customers’ emotions and determining
the position of the emotions on the core affect. Bespoke
tailoring evoked various positive and negative emotions. Theemotions extracted from the bespoke tailoring were confirmed
to evoke specific emotions that do not exist in Russell’s
emotion model [12], such as worry and expectation. We also
determined the position of the emotions on the core affect andclarified four clusters: pleasure-activation (Cluster 1), pleasuredeactivation (Cluster 2), displeasure-activation (Cluster 3),
and displeasure-deactivation (Cluster 4). The representative
emotions of Cluster 1 were uplifting and interesting. The
representative emotions of Cluster 2 were confusion and anxiety. The representative emotions of Cluster 3 were hesitanceand regret. The representative emotions of Cluster 4 weresatisfaction and feeling good.

[1] S. Xiao and I. Benbasat, “The formation of trust and distrust in recommendation agents in repeated interactions: a process-tracing analysis,”
Proceedings of the 5th international conference on Electronic commerce,
pp. 287–293, 2003.
[2] J. Cassell and T. Bickmore, “External manifestations of trustworthinessin the interface,” Communications of the ACM, vol. 43, no. 12, pp.
50–56, 2000.
[3] H. Honda, R. Hisamatsu, Y. Ohmoto and K. Ueda, “Interaction in a
Natural Environment: Estimation of customers’ Preference Based on
Nonverbal Behaviors,” Proceedings of the Fourth International Conference on Human Agent Interaction, pp. 93–96, 2016.
[4] C. Voeffray, “Emotion-sensitive human-computer interaction (HCI):State of the art-Seminar paper,” Emotion Recognition, pp. 1–4, 2011.
[5] Y. Ono, S. Aoyagi, M. Sugimoto, Y. Yamazaki, M. Yamamoto and N.
Nagata, “Application of classification method of emotional expression
type based on laban movement analysis to design creation,” InternationalConference on Human-Computer Interaction, pp. 143–154, 2019.
[6] M. Sugimoto, M. Yamamoto and N. Nagata, “The Emotions Evoked
during Individual Voluntary Manufacture: Their Characteristics and theTiming of Their Evocation,” Journal of Human Interface Society: humaninterface, vol. 21, no. 1, pp. 85–96, 2019 (in Japanese with Englishabstract).
[7] S. Imai, M. Sugimoto, K. Katahira, Y. Yamazaki, N. Nagata, A. Masuda,
K. Iwata and H. Uchiyama, “Indexing of riders’ emotion on motorcycle
based on core affect model Gathering and classifying of emotional
words ,” IEICE Technical Report, vol. 19, pp. 123–126, 2017 (inJapanese with English abstract).
[8] M. Sugimoto, Y. Yamazaki, F. Zhang, S. Miyai, K. Obata, M. Yamamoto and N. Nagata, “Differences in Customers’ Interactions withExpert/Novice Salesclerks in a Bespoke Tailoring Situation: A Case
Study on the Utterances of Salesclerks,” International Conference on
Human-Computer Interaction, pp. 131–137, 2019.
[9] J. A. Russell, “Core affect and the psychological construction ofemotion,” Psychological review, vol. 110, no. 1, pp. 145, 2003.
[10] J. Sanui, “Visualization of users’ requirements: Introduction of theEvaluation Grid Method,” Proceedings of the 3rd Design & Decision
Support Systems in Architecture & Urban Planning Conference, vol. 1,pp. 365–374, 1996.
[11] Y. Onoue, N. Kukimoto, N. Sakamoto and K. Koyamada, “E-Grid: a visual analytics system for evaluation structures,” Journal of Visualization,
vol. 19, no. 4, pp. 753–768, 2016.
[12] J. A. Russell, “A circumplex model of affect,” Journal of personality
and social psychology, vol. 39, no. 6, pp. 1161, 1980.
[13] P. Ekman and W. V. Friesen, “Constants across cultures in the face andemotion,” Journal of personality and social psychology, vol. 17, no. 2,
pp. 124–129, 1971.