Gaze distribution of an observer while imagining
wearing clothing portrayed in an advertisement
and predicting the impression on others
Fuyuko Iwasaki, Michiko Inoue, Masashi Nishiyama
Tottori University (Japan)
INTERNATIONAL CONFERENCE ON ENGINEERING PSYCHOLOGY AND COGNITIVEERGONOMIC (EPCE), HCII2024
1

Introduction (1/2)
Image-sharing social networking services (SNSs) have become a common platform
for advertising and purchasing clothing.
SNS feeds show many images of people wearing fashionably coordinated clothing.
We refer to the people in the images as subjects.

The subjects in SNS images are often photographedfrom head to toe, or their heads are hidden.

Whole-body image: a subject is visible from head to toe.
Head-occluded image: a subject is visible from shoulder to toe.
2Introduction (2/2)
People using SNSs subjectively predict whether they would make a good impression onothers in a certain outfit by imagining themselves wearing the portrayed subject's clothing.
We refer to people who use SNSs as observers.
When observers who are using SNSs makea prediction about the impression on others
that a certain outfit would produce,
they look directly at the subject's body.

✓ Imagine wearing a subject’s clothing✓ Predict the impression on others
Maybe I can make a good
impression on others
by wearing this clothing!?

While imagining wearing the subject's clothing,
it is likely that the observer's gaze is drawn to
the subject's body parts, which could provide
cues about the impression on others.

✓ GazeObserver

Subjects in SNS images3

Positioning of this study
Goal:
We investigated the gaze of female observers who were asked to imagine
wearing a subject's clothing and to predict the impression on others.

✓ GazeAlthough the analysis conditions differed from those in our present research, several previous studies
investigated the subject body parts that were most frequently gazed at by observers
who were asked to predict the degree to which others would find the subject attractive.

Previous studies:
Gervais et al. examined gaze characteristics for subjects with different body shapes in a whole-body image.
[Sex Roles, 2013]

Cornelissen et al. examined gaze characteristics for naked subjects in a head-occluded image.
[Evolution and Human Behavior, 2009]

Issue:
Previous studies did not consider the gaze of observers asked toimagine wearing a subject's clothing and to predict the impression on others.
4Purpose
We attempted to determine which body parts in a whole-body image
or a head-occluded image were the focus of a female observer
who was asked to imagine wearing the portrayed subject's clothing andto predict the impression on others.
Whole-body images

1. Gaze measurement

2. Gaze distribution analysis

Observer[3x]
Eye tracker

Subjects
Body parts detection

Body parts

Left toes

Head-occluded images

HeadChestObserver

Probability

Stimulus image

5

Hypothesis
To investigate the degree to which the observer's gaze was focused on each subject’s
body part, we asked the female observer to imagine wearing the portrayed subject’s
clothing and to predict the impression on others.
H1: When a female observer imagines wearing a subject's clothing
in a whole-body image and predicts the impression on others,
the observer's gaze will be most focused on the subject's head.

H2: When a female observer imagines wearing a subject's clothing
in a head-occluded image and predicts the impression on others,
the observer's gaze will be most focused on the subject's stomach.
6

Experimental condition 1
Stimulus images:

The same number ofstimulus images
for both formal andcasual clothing• The total number of stimulus images per observer was 104,
with 52 whole-body images and 52 head-occluded images.
• We used only the subject's standing posture,
where her clothes and face were visible.

Whole-body images

Observers:• We recruited 24 female observers(21.5±1.3 years old, university students, Japanese ethnicity).

Question:Above thetip line ofa chinHead-occluded images• We asked a female observer the following question when she viewed the subject in the stimulus image.
Q: When you imagine yourself wearing the subject's clothes,
do you feel that the people around you will have a good impression of you?
Each observer responded on a 4-point scale (4: yes, 3: likely yes, 2: likely no, 1: no).
7

Experimental condition 2
Setting:

Eye tracker

Observer’s eyes

Display110-120 cm

123 cm65 cm• 24-inch display (1920 x 1080 pixel resolution)• Stationary eye-tracker (gazepoint GP3 HD, 150 Hz)• We performed resampling using linear interpolation at 50 Hz
to ensure that the time intervals were equally aligned.

Procedure:

Table
70 cm

Measuring gazes• Start image (2 seconds) → stimulus image (3 seconds) → end image (2 seconds)• We asked an observer to look at the randomly placed white cross in the start image to avoid center bias.

2D heatmap
of measured gazes
[3x]

[3x]
8

Gaze analysis
Purpose:When analyzing the observer's gaze, we need to notice that the pixels
representing the body parts differ among the stimulus images.
The subject's body parts
are not aligned.

Body parts

Left toes

This probability was calculated based on the distance
between a pixel representing each subject body partand a pixel where the observer's gaze was measured.

HeadChest

Solver:[Kinoshita+, HCII, 2021]
We used the body-part attention probability analysis,
which indicates how likely it was that the observer's gaze wasfocused on each subject body part.

ProbabilityIssue:
When using a heatmap representation on stimulus images for gaze analysis,
it is not easy to compare the measured gaze pixels among stimulus images directly.

9Body parts for gaze analysis
𝑏1 : Head𝑏4 : Right shoulder

𝑏2 : Chest
𝑏5 : Left shoulder⚫ Whole-body image: 14 body parts
𝐵𝑤 = 𝑏1 , 𝑏2 , ⋯ , 𝑏14

𝑏6 : Right elbow

𝑏7 : Left elbow

𝑏8 : Right wrist

𝑏9 : Left wrist

𝐵ℎ = 𝑏2 , 𝑏3 , ⋯ , 𝑏14

𝑏10 : Waist

Excluding the head 𝑏1

𝑏3 : Stomach

𝑏11 : Right knee𝑏13 : Right toes

⚫ Head-occluded image: 13 body parts

𝑏12 : Left knee

𝑏14 : Left toes

When performing gaze analysis, we detected the body part positions using OpenPose.
10

Body-part attention probability

Stimulus image 𝐼𝑤

Observer 𝑜This probability indicates how likely it was that the observer's
gaze was focused on each subject body part.
The second term means theprobability that the gaze of anobserver is focused at a pixel 𝒊𝑤 .

𝒊𝑤 ∈𝐼𝑤∑ = diag(𝜎 2 , 𝜎 2 )෍ 𝑝 𝑏𝑤 𝑜, 𝐼𝑤 = 1𝑏𝑤 ∈𝐵𝑤

𝐵𝑤 = 𝑏1 , 𝑏2 , ⋯ , 𝑏14

Body parts[Kinoshita+, Body-part Attention Probability for Measuring Gaze During Impression Word Evaluation, HCII2021]

Left toes

Body-part
attention probability

Visualization of the body-partattention probability as a bar graph.

HeadChest𝑝 𝑏𝑤 𝑜, 𝐼𝑤 = ෍ 𝒩(𝒊𝑤 |𝒊𝑏𝑤 , ∑)𝑝 𝒊𝑤 𝑜, 𝐼𝑤

A pixel 𝒊𝑏𝑤 isa body part𝑏𝑤 .

Probability 𝑝 𝑏 𝑜, 𝐼

The first term is a normal distribution.
The closer the distance from a pixel 𝒊𝑤
to a pixel 𝒊𝑏𝑤 , the higher value is recorded.

A pixel 𝒊𝑤 is
an elementof 𝐼𝑤 .

11Subjective assessments on hypothesis H1H1: The observer's gaze will be most focused on the subject's head in a whole-body image.

We calculated the marginalized probability 𝑝 𝑏𝑊
for whole-body images.

0.4
0.3

𝑝 𝑏𝑊

0.2

1=෍ ෍ 𝑝 𝑏𝑤 |𝑜, 𝐼𝑤
𝑂𝑁

0.1
0

All whole-body images 𝐼𝑤
All observers 𝑜Head
Chest
StomachRight shoulder
Left shoulder
Right elbow
Left elbow
Right wrist
Left wrist
Waist
Right knee
Left knee
Right toes
Left toes

Body-part attention
probability 𝑝 𝑏𝑤

0.5…
12

𝑂=2412

𝑁=52

H1 did not hold. Instead, in the case of whole-body images, the female observer's gazetended to be focused most on the chest, followed by the head.
12

Subjective assessments on hypothesis H2
H2: The observer's gaze will be most focused on the subject's stomach in a head-occluded image.

We calculated the marginalized probability 𝑝 𝑏ℎ
for head-occluded images.

0.4

0.3

𝑝 𝑏ℎ

0.2

1
=෍ ෍ 𝑝 𝑏ℎ |𝑜, 𝐼ℎ
𝑂𝑁

0.1
0

All observers 𝑜Chest
StomachRight shoulder
Left shoulder
Right elbow
Left elbow
Right wrist
Left wrist
Waist
Right knee
Left knee
Right toes
Left toes

Body-part attention
probability 𝑝 𝑏ℎ0.5

All head-occluded images 𝐼ℎ…
1

2

𝑂=24

1

2

𝑁=52

H2 did not hold. Instead, in the case of head-occluded images, the female observer's gazetended to be focused most on the chest, followed by the stomach.
13

Gaze distribution for each stimulus image
We visualized how the gaze was focused on body partsfor each stimulus image.

Observer 𝑜
A pixel 𝒊𝑤 is
an elementof 𝐼𝑤 .

We generated heatmaps using the probability 𝑝 𝒊𝑤 𝑜, 𝐼𝑤 .
FocusedThe gaze was focused most on the subject's chest,
followed by the head.

Not focused

Stimulus image 𝐼𝑤

The gaze was focused most on the subject's chest,
followed by the stomach.
14Conclusions
We investigated which body parts were the focus of a female observer who was asked toimagine wearing a subject's clothing and to predict the impression on others.
The female observer's gaze would be focused most on the subject's chest,
followed by the head, in the case of whole-body images.

The female observer's gaze would be focused most on the subject's chest,
followed by the stomach, in the case of head-occluded images.
Future work

• We will investigate the observer's gaze distribution by measuring gaze diversity
concerning the gender, age, and ethnicity of observers.
• We will expand our analysis to various subjects' postures in stimulus images,
such as sitting and side-facing standing postures.
15