Heatmap overlay using
neutral body model for visualizing
the measured gaze distributionsof observers

M. Inoue, F. Iwasaki, M. Nishiyama, Y. Iwai
Tottori UniversityIntroduction
Eye-tracking researchers measure and compare the gaze distributions of people, because the gaze
distributions indicate the behavior of the observers when they view the bodies of other people.
Measures

Compares
Views

Gaze

Researcher Observer

Researcher: Gaze distribution

Ex. We consider how to compare whether there are differences in the measured gaze distributionsof observers viewing an attractive subject.
• Both hands down
• Slender• One hand up• Muscular

The researchers need a visualization method that can directly compare the gaze distributions
among subjects, even when they have different body poses and body shapes.
2

Existing methods
 2D heatmaps are often superimposed on images to visualize
where an observer's gaze focuses on a subject. [L. Nummenmaa 2012], [K. R. Irvine 2019]✘ A visualization that superimposes 2D heatmaps is time-consuming
for the researchers.

ComparesEx. The researchers must pay attention to the different poses of the hands and arms
and the different body shapes of the man and woman.
Researcher Other visualization methods have been proposed in which 3D heatmaps representing
the gaze distributions are superimposed on the surface of artificial objects
by aligning the shapes of known objects.
[X. Wang 2016][R. Takahashi 2018]✘ Applying these methods to human subjects is restricting
because humans are non-rigid and have various body shapes in different poses.
3Purpose
We propose a visualization method that superimposes 3D heatmaps onthe surface of a neutral human body to compare the locations ofobserver gazes directly when the observers view subjects in images.

＋2D heatmap

Neutral human body model

Our 3D heatmap

We use a neutral human body model with a body pose and shape that are normalized among subjects.

 Our method allows eye-tracking researchers to directly compare the gaze distributions on
the common 3D human model, even if the body poses and shapes of the subjects differ.
4

Our visualization method
S1. Pixel attention probability
𝑝𝑝 𝒙𝒙𝑗𝑗 𝑜𝑜, 𝑡𝑡,

Represents how muchthe gazes focus oneach pixel 𝒙𝒙𝑗𝑗GazeTime 𝑡𝑡

Subject region
in the image

Observer 𝑜𝑜

Neutral human body model𝒗𝒗𝑗𝑗 = 𝑚𝑚 𝒙𝒙𝑗𝑗 ;3D position 𝒗𝒗𝑗𝑗
on the surfaceof the modelVertex 𝒗𝒗𝑙𝑙 ∈ 𝒱𝒱 of the model

2D positionof pixel 𝒙𝒙𝑗𝑗 in

S2. Vertex attention probability
Represents how much the gazes
𝑝𝑝 𝒗𝒗𝑙𝑙 𝑜𝑜, 𝑡𝑡,
focus on each vertex 𝒗𝒗𝑙𝑙 ofthe neutral human body model 𝒱𝒱
Marginalized by𝑜𝑜 ∈ 𝒪𝒪 and 𝑡𝑡 ∈ 𝒯𝒯

Set of observers 𝒪𝒪

𝑝𝑝 𝒗𝒗𝑙𝑙

S3. 3D heatmap overlay

Set of times

𝒯𝒯 = 1, ⋯ , 𝑇𝑇on the surface
Visualizes 𝑝𝑝 𝒗𝒗𝑙𝑙of the neutral human body model

Visualization images
using the 3D heatmap
5Experimental conditions
 We measured observer’s gaze distributions.
 We asked the question “Do you think that the subject’s hand are beautiful?”.
Stimulus images
 Observers answered the question “yes” or “no”.
 24 observers:
12 males, 12 females, 22.4 years old Setting:Observer110 - 120cm

Eye tracker

Display
65 cm 100 cm
73 cm

Subject 1

Subject 2

Subject 3

 We evaluated the effectiveness of our method using visualization images.
 𝑀𝑀2𝑑𝑑 (Conventional method): 2D heatmap representing the pixel attention probability
 𝑀𝑀3𝑑𝑑 (Our method): 3D heatmap representing the vertex attention probability

6Visualization results
𝑀𝑀2𝑑𝑑

2D heatmap

Subject 1

Attention probability:
Low

Subject 2High

Subject 3✘ Eye-tracking researchers must consider the differences in the body poses and shapes
because of the variation in the body alignment among subjects.

𝑀𝑀3𝑑𝑑

3D heatmap(Ours)

Subject 1

Subject 2

Subject 3

 Eye-tracking researchers only need to compare the same positions on the surface ofthe model because these 3D heatmaps make the body alignment equal for all subjects.

The gaze distributions reveal the gazes most often focus on the hand, followed by the head.
We can reach this conclusion more directly using our method 𝑀𝑀3𝑑𝑑 than the conventional method 𝑀𝑀2𝑑𝑑 .

7

Conditions of subjective assessment
 We conducted a subjective assessment to determine whether the visualization images
generated by 𝑀𝑀2𝑑𝑑 or 𝑀𝑀3𝑑𝑑 enable the gaze distributions for various subjects to be directlycompared.
 16 young eye-tracking researchers: 13 males, 3 females, 23.4 years old We asked the researchers to choose the visualization images they felt would better
directly facilitate the comparison of gaze distributions among the subjects.
 The researchers replied with one of the following answers: 𝑀𝑀2𝑑𝑑 , 𝑀𝑀3𝑑𝑑 , or neutral.
 Setting:

Example of the visualization images on the display

Eye-tracking
researcher
Display
65 inch150 cm90 cm
𝑀𝑀2𝑑𝑑 : 2D heatmap

𝑀𝑀3𝑑𝑑 : 3D heatmap

8Result of the subjective assessment
Conventional method 𝑀𝑀2𝑑𝑑 : 2D heatmap

NeutralOur method 𝑀𝑀3𝑑𝑑 : 3D heatmap

048

12

Number of eye-tracking researchers

16• Our method 𝑀𝑀3𝑑𝑑 obtained the highest results.
• Some researchers chose 𝑀𝑀2𝑑𝑑 because they were familiar with the 2D heatmaps,
which made it easy to identify the body parts focused on by gaze for each subject.
Our method 𝑴𝑴𝟑𝟑𝟑𝟑 enables eye-tracking researchers to compare differences more directlyin the gaze distributions than the conventional method 𝑴𝑴𝟐𝟐𝟐𝟐 ,even when the body poses and shapes are different.
9Conclusions
 We proposed a method of superimposing 3D heatmaps on the surface of a neutral human bodymodel to visualize where the gazes of observers focus when they view subjects in images.
 Eye-tracking researchers could directly compare the gaze distributions for the visualization images
obtained using our method, even when the body poses and shapes differed among subjects.
Compares

Measures
…

Views
Gaze
ObserverResearcher

Researcher

Future work
We will expand the evaluation for various shape characteristics, such as the weight of the subjects.
10