Subjective Scores and Gaze Distribution in
Personality Evaluations:
Effect of Subjects’ Clothing on
Observers’ Impressions of Them
Michiko INOUE1 ,Masashi NISHIYAMA

1[0000−0002−5964−3209], and Yoshio IWAI1

Graduate School of Engineering, Tottori University
101 Minami 4-chome, Koyama-cho, Tottori, 680-8550, Japannishiyama@tottori-u.ac.jp

Abstract. The clothing that coworkers and others wear in the workplace affects our evaluations of them, including what personality traits
we ascribe to them. In this study, we sought to determine whether observers’ subjective impressions of a subject’s sincerity and nervousness
vary depending on the subject’s clothing, and the extent to which observers look at subjects’ clothing while making those assessments. We
measured subjective scores for sincerity and nervousness as componentsof personality traits perceived by observers. Specifically, we displayed
stimulus images containing subjects wearing several types of clothing andmeasured observers’ subjective scores and gaze distributions. The experimental results indicated that the subjects’ clothing strongly affected theobservers’ subjective assessments of the subjects’ sincerity and nervousness. We also confirmed that the observers gazed at both the subjects’
clothing and head regions.
Keywords: Clothing · Personality evaluations · Subjective scores · Gaze.

1

Introduction

In online-interaction interfaces, observers frequently form positive impressions ofsubjects in images displayed on screens, and also form unconscious impressionsof those subjects. In particular, observers’ impressions are strongly affected by
the subjects’ clothing (e.g., a woman’s dress) [7]. Consider a female shopper
exploring an apparel shopping website to purchase clothing to wear at the office.
Figure 1 shows that she is concerned about what impression her clothing will
give to observers; here, we consider that she is imagining herself wearing what
the subjects in the images are wearing. In this paper, we report on our study on
how observers perceive subjects wearing office-style clothing. We assumed thatthe observers were unfamiliar with the subjects.
We consider that the most important impressions will vary depending on individual contexts; however, many relevant impressions are formed in interpersonal

2

M. INOUE et al.

Subject

User

Coat

Dress
Subject

She isprobablysincere.

Maybe
she issincere.

ObserverObserverFig. 1. A user buying clothes is concerned about what impression they will give toobservers.

relationships. For example, physical appearance is relevant to the impressions
that are formed of employees who interact with the public face-to-face, suchas retail clerks or ushers at trade shows. In contrast, personality traits suchas sincerity and nervousness are relevant to the impressions that are formed ofengineers in IT companies or of human resource personnel. In this study, we
considered observers’ impressions of others’ personality traits in an office workplace. More specifically, we investigated sincerity and nervousness as components
of one of the Big Five personality traits, conscientiousness and neuroticism [6].
The Big Five are the most commonly used basic personality traits.
Our aim was to quantitatively measure a subjective sincerity score and a
subjective nervousness score from Japanese participants. Perceptions of others’
sincerity and nervousness vary depending on various subjective elements, suchas their hairstyle, facial expressions, and clothing. In this study, we focus on
subjects who are wearing clothing for working in an office. In one study [5], it
was reported that work clothing affects perceived personality traits. However,those authors used only a text-based questionnaire for analysis. No researcher
has yet examined whether what people are wearing in images affects others’
impressions of them. To partially close this research gap, we investigated thefollowing hypothesis in the present study:
Main hypothesis (Hm ): Observers’ subjective scores for a subject’s sincerity
and nervousness will vary depending on what the subject is wearing.
Furthermore, we investigated whether observers viewed subjects’ clothes while
making their assessments. It is generally well-known that an observer’s gazetends to fixate on the subject’s head for a long duration [2]. On the basis of thisknowledge, then, we investigated the following hypothesis:
Sub hypothesis (Hs ): Observers will gaze at a subject’s clothing as well as
their head while making assessments about the subject’s sincerity and nervousness.
Experimental results showed that both hypotheses Hm and Hs were supported.
We confirmed that the subjects’ clothing strongly affected the observers’ subjective assessments of the subjects’ sincerity and nervousness. We also confirmed

Subjective Scores and Gaze Distribution in Personality Evaluations

𝐶𝑐 : Coat

𝐶𝑑 : Dress3

𝐶𝑠 : Sleepwear

Fig. 2. The clothing categories of coat Cc , dress Cd , and sleepwear Cs used in ourinvestigation.

that the observers gazed at both the subject’s clothing and head regions while
making their assessments.

2

Methods for measuring subjective scores and gaze
distributions during personality evaluations

2.1

Stimulus images

To investigate the hypotheses Hm and Hs , we selected stimulus images froma multiview clothing (MVC) dataset [9]. The MVC dataset consists of 161,260
images comprising 264 clothing category labels, such as button-up shirts, polos, tank tops, swimwear, and sweaters. Each stimulus image contained a person
wearing clothing from one of the clothing categories, and was labeled accordingly(e.g., “tops and coats”). We selected only images of women in which the entire
body was included. Furthermore, we selected only clothing category labels belonging to whole-body items such as dresses and coats, and eliminated categories
containing exclusively upper- or lower-body items (e.g., tops or bottoms). The
following labels were assigned to the clothing categories:
Clothing category Cc : Coat,
Clothing category Cd : Dress,
Clothing category Cs : Sleepwear.
Figure 2 shows examples of stimulus images for coat Cc , dress Cd , and sleepwear
Cs . The reasons for selecting these clothing categories are described below. We

4

M. INOUE et al.

𝐶𝑐 : Coat

𝐶𝑑: Dress 𝐶𝑠: Sleepwear

Fig. 3. Average calculated from the stimulus images for each clothing category.

used coats Cc that are frequently worn when commuting to workplaces such as
IT companies and to jobs such as human resources in Japan, where observers’
impressions of personality traits are particularly important. We used a dress Cd
typical of those worn by sales clerks at workplaces such as retail department
stores and trade shows, where a person’s appearance is especially salient. We
used sleepwear Cs as a simple comparison with the other clothing categories.
We resized the height of all the stimulus images to 960 pixels while maintaining the aspect ratio of the original MVC dataset images. To control theexperimental conditions, we used only the frontal orientation of the subject’s
body. Furthermore, each subject appeared in multiple images and wore different
clothes in each. The number of stimulus images for each clothing category was
24 and the total number of stimulus images was 72. We checked the alignment of
all the images using the average image computed from all the stimulus images.
Figure 3 shows the averaged stimulus images for each clothing category. The
positions of the face and torso were deemed to be in rough alignment, while thepositions of the wrist and toes were not in alignment.

2.2

Personality traits

We selected the adjectives to represent personality traits in the present study
by referencing existing analytical studies [1, 10] that employed the Big Five personality traits. These traits are categorized as [6]: openness to experience, conscientiousness, extraversion, agreeableness, and neuroticism. Table 1 lists theadjectives that represent the factors of each of the Big Five personality traits.
To detect differences in subjective scores, we used the contrasting categories
of conscientiousness and neuroticism. Conscientiousness indicates a tendency
toward responsibility, diligence, and seriousness, all of which are strongly associated with the workplace and specifically that found in many IT companies
and in departments such as human resources. In contrast, neuroticism indicatesa tendency toward strong stress tolerance, another feature associated with theworkplace.

Subjective Scores and Gaze Distribution in Personality Evaluations

5

Table 1. Examples of adjectives representing the Big Five personality traits.
Trait
Adjectives
Openness to experience Curious, Creative, Imaginative, Smart
Conscientiousness
Sincere, Diligent, Conscientious, Cautious
Extraversion
Fun, Active, Enthusiastic, Social
Agreeableness
Friendly, Generous, Kind, Forgiving
Nervous, Emotional, Impulsive, Stressful
Neuroticism

Finally, we selected the following adjectives to represent conscientiousness
and neuroticism, respectively.
(a): Sincere(b): Nervous

2.3

Body-part attention probability for analyzing gaze distribution

To investigate the sub hypothesis Hs , we measured the body-part attention
probability [8] to validate where the observer’s gaze fixated. The body-part attention probability indicated how long an observer viewed which of the subject’s
body parts in the stimulus image during gaze measurement. This allowed us todirectly compare the body-part attention probabilities among all the stimulus
images, which contained a variety of subject postures. For example, even though
the subjects’ hand postures differed among the stimulus images (e.g., the hand isin the pocket or the hand is raised), we could quickly check whether observers’gazes frequently fixated on the hands by viewing the values of the body-partattention probabilities.
To calculate the gaze attention probability, we used the procedure proposedby Kinoshita et al. [8]. Figure 4 shows the body parts used in our gaze analysis.
We included 12 body parts: the nose of the face region, the left and right shoulders, the waist of the torso region, the left and right elbows, the left and rightwrists, the left and right knees, and the left and right toes. All the body parts
except the nose comprised the clothing region. Figure 5 shows examples of 12
body parts automatically detected by OpenPose [4].
2.4Experimental conditions

Twenty-four observers (13 men, 11 women; average age 21.9 years) participatedin our experiment. We fully explained the limitations of gaze measurement to
the participants and obtained their written consent. Figure 6 shows the experimental setting. The observer was seated at 65 cm from the display, and thenadjusted the chair height so that their eye level was 110–120 cm above the ground
while seated. The display size was 24 inches (resolution: 1920×1080 pixels, size:
53 cm × 30 cm). We used a Gazepoint GP3 HD eye tracker device to measureeye movement. The sampling rate was 150 Hz, and the angular resolution was

6

M. INOUE et al.

Nose
Right shoulder

Left shoulder

Right elbow
Right wrist

Left elbow
Left wrist
Waist

Right knee

Left knee

Right toes

Left toes

Fig. 4. The 12 body parts included in our gaze analysis. We used the nose of the faceregion, the left and right shoulders, the waist of the torso region, the left and rightelbows, the left and right wrists, the left and right knees, and the left and right toes.

𝐶𝑐 : Coat

𝐶𝑑 : Dress𝐶𝑠 : Sleepwear

Fig. 5. Examples of 12 body parts detected in the stimulus images for each subject’s
clothing category Cc , Cd , and Cs .

between 0.5 and 1.0 degrees. We displayed the stimulus image at a random position on the screen to avoid center bias [3]. Each observer viewed 24 stimulus
images, selected randomly from the repository of 72 stimulus images.
2.5

Procedure for measuring observers’ subjective scores

The observers were asked the following questions for each stimulus image:Qs : How sincere do you perceive the subject to be?
Qn : How nervous do you perceive the subject to be?
The observers gave their subjective rating for each image on a 4-point scale,ranging from 0 to 3. The higher the score, the more sincere or nervous theobserver perceived the subject to be. We used the following procedure to measureobservers’ scores for the questions.

Subjective Scores and Gaze Distribution in Personality Evaluations

7

65 cm
Display
100 cmParticipant
110–120 cm

Eye tracker
73.5 cm

Fig. 6. Experimental setting. Twenty-four observers participated in our experiment.
We used a screen to display the stimulus images. To measure gaze distribution, we useda gaze measurement device with a 150-Hz sampling rate.

*
*

*
*

3

* : p < .05

*
3

2

2

1

1

0

𝐶𝑐 : Coat

𝐶𝑑 : Dress 𝐶𝑠 : Sleepwear0** : p < .05

𝐶𝑐 : Coat(a) 𝑄𝑠 : Sincere

𝐶𝑑 : Dress 𝐶𝑠: Sleepwear(b) 𝑄𝑛 : Nervous

Fig. 7. Subjective scores relating to the main hypothesis Hm . In (a), we asked thequestion Qs to measure perceived sincerity. In (b), we asked the question Qn to measure perceived nervousness. We confirmed that the subjects’ clothing (Cc , Cd , and Cs )strongly affected the observers’ subjective assessments of their sincerity and nervousness.

P1 : We randomly selected one observer.
P2 : We set the question as either Qs or Qn .
P3 : We randomly selected one stimulus image.
P4 : We displayed a gray image presented on the screen for 2 seconds.
P5 : We displayed the stimulus image for 8 seconds while measuring the gazemovement.
P6 : We displayed a black image for 3 seconds, and the observer answered thequestion verbally.
P7 : We repeated the procedure P3 to P6 for 24 stimulus images.
P8 : We repeated the procedure P2 to P7 for the total number of observers.

3

Experimental results

3.1

Results for the main hypothesis Hm

Figure 7 shows the subjective scores for the main hypothesis Hm . In this figure,(a) shows the results for sincere Qs and (b) shows the results for nervous Qn . It isclear that the subjective scores for coat Cc were highest and those for sleepwear

8

M. INOUE et al.

60

57.2

Nose

40

10
0

Nose

Left shoulder
Waist

Waist

11.0
7.1
4.4
4.2 1.6
3.0 4.30.54.21.2
1.3

𝐶! : Sleepwear
49.4

Nose
Left shoulder

30
20

𝐶# : Dress50.7

50Frequency

𝐶" : Coat

5.6

11.5

9.6
3.9 4.6
2.3 3.9
2.6
2.4
1.0 1.9

Left shoulder
Waist
14.5
5.6
4.2 10.6
1.7
2.1 2.51.34.41.7
2.0(a) 𝑄! : Sincere

60

Frequency

50
40
30
20
10
0

55.4

𝐶" : Coat𝐶# : Dress
55.1

Nose

𝐶! : Sleepwear
50.0

Nose

Nose

Left shoulder

Left shoulder

Left shoulder

Waist

Waist

Waist

10.9
7.7
4.7 4.5
5.5 1.9 4.0
0.2 0.9
2.7
1.6

7.1

10.0
6.2
3.9
5.7 5.2
2.6
0.5 1.0

1.91.810.2 11.1
3.0 4.1
3.3 3.9
2.9
2.8
1.0 1.6

6.1(b) 𝑄": Nervous
Fig. 8. Body-part attention probability (%) for clothing categories Cc , Cd , and Cs .
In (a), we asked the question Qs to measure perceived sincerity; in (b), we asked thequestion Qn to measure perceived nervousness. All the body parts except for the nose
corresponded to the clothing region. Observers viewed both the subjects’ head regions
and clothing regions.

Cs were lowest for both questions Qs and Qn . Furthermore, the subjective scoresfor coat Cc in (a) and (b) were higher than the neutral score of 1.5. Additionally,
the subjective scores for dress Cd and sleepwear Cs were lower than the neutralscore of 1.5.
Next, we checked for significant differences in the subjective scores between
the clothing categories, Cc , Cd , and Cs , using the Steel-Dwass test for multiple
comparisons. Results showed significant differences (p < .05) in subjective scoresamong all the clothing categories for both questions Qs and Qn . We confirmedthat the subjects’ clothing strongly affected the observers’ subjective assessmentsof their sincerity and nervousness. The results indicated that the observers perceived the subjects as more sincere and nervous when the subjects wore coatsrather than dresses and sleepwear.
3.2

Results for the sub hypothesis Hs

Figure 8 shows the results of the body-part attention probability analysis. As
described in Section 2.3, the region containing all the body parts except for
the nose corresponds to the clothing region. The observers’ gazes were fixedSubjective Scores and Gaze Distribution in Personality Evaluations(a) 𝐵𝑤 : Whole body(b) 𝐵𝑇 : Torso and legs

9(c) 𝐵𝐻 : Head

Fig. 9. Examples of the stimulus images used in the additional evaluation.

on the subjects’ head region (i.e., containing the nose) for the most prolonged
time duration, and on the subjects’ clothing regions for the remaining time. We
observed this same tendency for all clothing categories. From these results, we
can confirm that the observers gazed at both the subjects’ clothing and headregions. Therefore, we believe that Hs is supported.

4

Additional evaluation

We further evaluated observers’ responses to conditions in which the subjects’
head regions (where observers’ gazes fixate the longest) and clothing regions werealternately either visible or not visible. We set the stimulus image conditions as
BW : Whole body,
BT : Torso and legs,
BH : Head.
Figure 9 shows the examples of the stimulus images. In condition BW of (a)in this figure, the subject’s whole body is visible, the same condition as thatshown in Figure 2. In condition BT of (b), the top 20% of the stimulus image
is obscured with the background color. In condition BH of (c), the lower 80%of the stimulus image is obscured with the background color. The experimental
conditions, except for the stimulus images, were the same as those described in
Section 2.
Figure 10 shows the results for the subjective assessments of the whole body
BW , the torso and legs BT , and the head BH for both questions (i.e., regarding
perceived sincerity Qs and perceived nervousness Qn ). Note that the results for
BW shown in this figure are the same as those shown in Figure 7. A comparisonof whole body BW and torso and legs BT showed that the ranking order ofthe subjective scores for coat Cc , dress Cd , and sleepwear Cs was the same.
By contrast, a comparison of whole body BW and head BH showed that the
ranking order of the subjective scores for clothing categories Cc , Cd , and Cs
differed. For questions about perceptions of sincerity Qs when torso and legs BT

10

M. INOUE et al.

*

3

*

*
: p < .05

* *: p < .05

3

*

*

*
* : p < .05

2

2

2

1

1

1

0

𝐶# : Dress𝐶" : Coat

𝐶! : Sleepwear

0

𝐵# : Whole body

3

*

*

𝐶" : Coat𝐶# : Dress𝐶! : Sleepwear0**: p < .05

3*
* : p < .05

2

1

1

1

𝐵# : Whole body0

𝐶" : Coat𝐶# : Dress𝐶! : Sleepwear

3

2

𝐶# : Dress 𝐶! : Sleepwear

𝐶# : Dress

𝐵": Head2

𝐶" : Coat

𝐶" : Coat

𝐵! : Torso and legs(a) 𝑄! : Sincere* :* p < .05

0* : p < .05

3

𝐶! : Sleepwear

𝐵! : Torso and legs(b) 𝑄": Nervous

0

𝐶" : Coat𝐶# : Dress 𝐶! : Sleepwear

𝐵": HeadFig. 10. Subjective scores for the stimulus image conditions of whole body BW , torso
and legs BT , and head BH used in the additional evaluation. In (a), we asked thequestion Qs to measure perceived sincerity. In (b), we asked the question Qn to measure perceived nervousness. All three clothing categories, Cc , Cd , and Cs , were used.
By comparing this figure with Figure 10, we could conclude that the stimulus images
containing the subject’s torso and legs, which represent the clothing region, significantly affected the observer’s subjective scores compared with those containing only
the subject’s head region.

was displayed, the subjective score for coat Cc exceeded the neutral score of 1.5.
For questions about perceptions of nervousness Qn when torso and legs BT was
displayed, the subjective scores for Cc , Cd , and Cs did not exceed the neutralscore of 1.5. For questions about perceptions of sincerity Qs when head BH wasdisplayed, the subjective scores for Cc , Cd , and Cs were higher than the neutralscore of 1.5. For questions about perceptions of nervousness Qn when head BH
was displayed, the subjective scores frf Cc , Cd , and Cs were close to the neutralscore. These results illustrate that the stimulus images containing the subject’s
torso and legs, which represented the clothing region, significantly affected theobservers’ subjective scores compared with those containing only the subject’s
head region.

5

Conclusions

In this study, we investigated the influence on observers of the type of clothing
being worn by subjects in digitally presented stimulus images. More specifically,observers rated their subjective impressions of the subjects’ personality traits

Subjective Scores and Gaze Distribution in Personality Evaluations

11

in response to images showing subjects wearing a coat, a dress, or sleepwear.
Both the main hypothesis Hm and the sub hypothesis Hs , respectively, were
supported: observers’ perceptions of a subject’s sincerity and nervousness varied
depending on what the subject was wearing, and observers gazed at both theclothing and the face of the subjects while making their assessments.
In future work, we will expand our investigation by using more adjectives that
represent the Big Five personality traits, and using more clothing categories.

Acknowledgment
This work was partially supported by JSPS KAKENHI Grant No. JP20K11864.

References
1. Ames, D.R., Bianchi, E.: The agreeableness asymmetry in first impressions: perceivers’ impulse to (mis)judge agreeableness and how it is moderated by power.
Personality and Social Psychology Bulletin 34, 1719–36 (2008)
2. Bareket, O., Shnabel, N., Abeles, D.: Evidence for an association between men’sspontaneous objectifying gazing behavior and their endorsement of objectifying
attitudes toward women. Sex Roles 81 (2019)
3. Bindemann, M.: Scene and screen center bias early eye movements in scene viewing.
Vision Research 50(23), 2577 – 2587 (2010)
4. Cao, Z., Hidalgo, G., Simon, T., Wei, S.E., Sheikh, Y.: Openpose: Realtime multiperson 2d pose estimation using part affinity fields. IEEE Transactions on PatternAnalysis and Machine Intelligence 43(1), 172–186 (2021)
5. Gillani, B.S., Haider, S.K., Jan, F.A.: The relationship of clothing with personal
identity of different clothing style personalities among working women. Journal ofManagement Info 3, 1–4 (2016)
6. Goldberg, L.R.: The development of markers for the big-five factor structure. Psychological assessment 4(1), 26–42 (1992)
7. Johnson, K., Lennon, S.J., Rudd, N.: Dress, body and self: research in the social
psychology of dress. Fashion and Textiles 1, 1–24 (2014)
8. Kinoshita, K., Inoue, M., Nishiyama, M., Iwai, Y.: Body-part attention probability
for measuring gaze during impression word evaluation. In: Proceedings of 23rd
International Conference on Human-Computer Interaction. pp. 105–112 (2021)
9. Liu, K.H., Chen, T.Y., Chen, C.S.: Mvc: A dataset for view-invariant clothing
retrieval and attribute prediction. In: Proceedings of the ACM International Conference on Multimedia Retrieval. pp. 313–316 (2016)
10. Nandi, A., Nicoletti, C.: Explaining personality pay gaps in the uk. Applied Economics 46(26), 3131–3150 (2014)