2020 年度

修士課程最終発表会

M23

VR 環境における視線の空間および過渡分布の数理解析に関する研究
Mathematical Analysis of Eye-gaze Space andTransient Distributions on VR Environment
Akira HIRAIMicheletto Laboratory
Graduate School of Nanobioscience,
Yokohama City University, Yokohama, Japan

Outline
1. Introduction
Background
Motivation
2. Experiment and Results
Searching target on test pattern
Clustered by K-Means and Elbow method
Classify using Probability Ellipse and Reaction time
Transient Analysis by Fourier Transformation
Neural Network Analysis of Fourier Transformed Data
3. Discussion
4. Summary

Introduction
Background

IssuesWhat’s the relationship between
eye-gaze moving and perception ?
What’s the mechanism ofVR sickness ?

The VR market is expected to grow significantly in the futureMotivation and Methodology
Motivation

Transient Analysis

My interest
VR Sickness

Eye-Gaze Space Analysis
Visual Search

At Micheletto LabMethodology

FOVE 0 VR

Eye-gaze tracking system

Outline
1. Introduction
Background
Motivation
2. Experiment and Results
Searching target on test pattern
Clustered by K-Means and Elbow method
Classify using Probability Ellipse and Reaction time
Transient Analysis by Fourier Transformation
Neural Network Analysis of Fourier Transformed Data
3. Discussion
4. SummarySearching target on test pattern
Eye-gaze tracking dataTarget

Fig. 1. Test pattern

Clustering

Elbow point

Fig.2. Clustered by K-Means and Elbow method

StareSearchingFig.3. ClusteredresultClustered by K-Means and Elbow method
K-Means method
Clustered data using given number of clusters

The objective function2KΣΣ

i=1 x∈Xix － μi

Elbow method
Optimization of number of clusters
using calculation of distortion

Elbow point

Probability Ellipse
Clarify parameters on “Probability Ellipse” to characterize clusters
Reaction time to find the targetτR= 16.1 sec.

Major axis ＝ 2σA

Center ofellipse

SearchingStare

Minor axis
＝ 2σB
中心座標2σA

2σB( 3.59, -3.56)

7.01

4.26(-2.43, -4.65)

1.78

1.15

Gaze-search Result
Characterization of Gaze-searchStare on Target

Characterization of “Stare”
and “Search” by key-parameter ρ.
Gaze Density
ρ＝Gaze Points
πσAσB
Searching
Stare on Selected PatternReaction Time
Relationship between Searching and Reaction TimeF distribution
Number (Wrong)

Number (Correct)

Histogram of Reaction
TimeF test ： P value
0.0235 < 0.05

Outline
1. Introduction
Background
Motivation
2. Experiment and Results
Searching target on test pattern
Clustered by K-Means and Elbow method
Classify using Probability Ellipse and Reaction time
Transient Analysis by Fourier Transformation
Neural Network Analysis of Fourier Transformed Data
3. Discussion
4. Summary

VR Sickness Test
Transient Analysis on Game Style Test
Stage-1
w/o Shake
Stage-2
w ShakeStage-3
w ShakeGame Style Test
Stage-4
w ShakeFourier Transform∞𝐹 ( 𝜔 )= ∫ 𝑓 ( 𝑡 ) 𝑒−2 𝜋 𝑖 𝜔 𝑡𝑑𝑡−∞

The Gaze data is obtained as ().
Fourier Transform of any frequency is𝑘 ∈𝑁sin ( 2 𝜋 ×5 𝑡 ) +𝐶𝑜𝑛𝑠𝑡𝑎𝑛𝑡

1−2 𝜋 𝑖 𝜔 𝑡𝐹 ( 𝜔 ) = ∑ 𝑥𝑘 𝑒
𝑁 𝑘Constant is not affecting Fourier Transform𝑘Transient Analysis
Transient Analysis by Fourier Transformation
Stage-1

Stage-2

Stage-3

Stage-4

Fourier Transformation of x-axis eye-moving
Stage-1

Stage-2

Stage-3

Stage-4

Neural Network(1)NeuronNeuron with synapse

Active function ： Sigmoid function𝑜 ( 𝑥 )=

1−𝑥1+𝑒𝑜 ′ ( 𝑥 )=𝑜 ( 𝑥 ) ( 1 − 𝑜 ( 𝑥 ) )Neurons connect and disconnect synapses(strengthening and weakening weighting)
Weighting parameter （ Synapse ）𝑥 𝒐(𝒙 )

𝑤𝑥=𝑤 ∙𝑜( 𝑥 )𝒙𝑜(𝑥)Neural Network(2)Neural Network Analysis of Fourier Transformed Data
Examinee H
Stg-1
Stg-2
Stg-3
Stg-4
Examinee F
Stg-1
Stg-2
Stg-3
Stg-4

Examinee T
Stg-1
Stg-2
Stg-3
Stg-4

Examinee O
Stg-1
Stg-2
Stg-3
Stg-4

Examinee M
Stg-1
Stg-2
Stg-3
Stg-4

Analysis Result
Neural Network Analysis and VR sickness
Examinee T, O, F and M
Stg-1
Stg-2
Stg-3
Stg-4

Correction rate
13.9%
35.7%
38.7%
35.1%

Response of Examinees (Feeling of VR Sickness by Level 0-5)
Examinee
Stg-1
Stg-2
Stg-3
Stg-4

H0321T0122O0231F0222M0211Ave.
0.0
2.0
2.0
1.4

→ The Neural Network System detected essential eye movements
related VR sickness without individual differenceDiscussion(1)Clustered by K-Means and Elbow method
Classify using Probability Ellipse and Reaction time We achieved Auto-Clustering using K-Means and Elbow method
 We also achieved to classify “Stare” and “Searching” by Gaze
Density and Reaction Time These analysis methods are first proposed by us We need other approaches for Transient Analysis

Discussion(2)Neural Network Analysis and VR sickness
 Background View and Shaking induces VR sickness
 Neural Network Analysis of eye-gaze moving can detect essential factors ofVR sickness
 Eye-gaze movement have relationship to VR sickness
 Further Verification is necessary to determine which Factors cause VRsickness
Large Signal

Small Signal

Stage-1

Large Signal
Moderate Signal

Moderate Signal

Stage-2

Stage-3Moderate Signal

Stage-4

0.05

Summary・ We obtained eye-gaze vector and coordinates data on
FOVE 0 gaze-tracking platform・ We characterized behavior of the eye-gaze vector andobtained characteristic movements of “Stare” and “Search”
・ We proposed key-parameters to characterize gaze-moving
・ We also proposed Fourier Transformation and Neural Network techniquefor Transient gaze-moving analysis
・ We obtained positive correlation on results of Neural Network analysis
for VR sickness
・ I had an oral presentation at VISION 2021 winter conferenceComparison of the Fourier spectrum
The height of the peakis different.
Stage1

Stage4

Correlation from 20Hz
to 80Hz is important fordiscrimination

Stage2

Saccade is 3Hz to 30Hz(VISION 2021 winterconference)

Stage3
There is a peak ornot.

0.05

Test （ Feedforward)𝑖⃗
𝑥1𝑜1⃗h𝑤11𝑥1𝑜1

𝑤1 2𝑥2𝑜2 𝑤
𝑥3𝑜3

𝑊1

13𝑥2𝑜2

𝑊2𝑤11

Active Function : Sigmoid Function

𝑤1 2𝑥11𝑜 ( 𝑥 )=−𝑥1+𝑒𝑤1 3𝑥2𝑜 ′ ( 𝑥 )= 𝑜 ( 𝑥 ) ( 1 − 𝑜 ( 𝑥 ) )𝑥3𝑜3 𝑤1 4𝑥 4𝑜4⃗𝑜𝑥3⃗h=𝑊 1 𝑜 ( 𝑖⃗ )⃗𝑜=𝑊 2 𝑜 ( h⃗ )Train （ Backpropagation ）⃗h𝑊1𝑖⃗𝑤11 𝑥 𝑜
1 1𝑥1𝑜1

𝑤21𝑥2𝑜2

𝑤31𝑥3𝑜3

𝑤41

from the nextlayer𝜕𝐸 ⃗ 𝑇
⃗
=𝛿𝑜
𝜕𝑊

𝑥2𝑜2
𝑥3𝑜3

𝑊2𝑤11𝑤21𝑥3

Teaching Vector

21 ⃗⃗ )|𝐸 = |𝑡 − 𝑜 ( 𝑜2𝑊 ←𝑊 +𝜂𝑥1

𝑤31 𝑥2𝑥 4𝑜4

𝜕𝐸
𝜕𝑊

Output to Hidden

Hidden to Input （ In general)𝜕 𝐸 𝜕 𝐸 𝜕 𝑜 𝑖 𝜕 𝑥𝑖=∙∙
𝜕 𝑤 𝑖𝑗 𝜕 𝑜𝑖 𝜕 𝑥 𝑖 𝜕𝑤 𝑖𝑗
𝜕𝐸¿∙ 𝑜′ 𝑖 ∙ 𝑜 𝑗
𝜕 𝑜𝑖

𝜕 𝐸 𝜕 𝐸 𝜕 𝑥𝑖=∙
𝜕 𝑤 𝑖𝑗 𝜕 𝑥 𝑖 𝜕 𝑤𝑖𝑗¿ ( 𝑡 𝑖 − 𝑜𝑖 ) ∙ 𝑜 ′ 𝑖 ∙𝑜 𝑗Use

¿ ( 𝑊 𝑛𝑒𝑥𝑡 𝑇 ⃗𝛿𝑛𝑒𝑥𝑡 ° ⃗𝑜′ ) ⃗𝑜𝑇⃗𝑜Error function(Neural Network is acquiring weighting matrix forminimizing an error function.

)

∙𝑜 𝑗

¿¿

¿ 𝛿𝑖 ∙ 𝑜 𝑗
𝜕𝐸𝛿𝑖 =𝜕 𝑥𝑖

𝑛𝑒𝑥𝑡𝜕 𝐸 𝜕 𝑥𝑘
¿ ∑∙𝑛𝑒𝑥𝑡
𝜕 𝑥𝑖𝑘 𝜕 𝑥𝑘

𝑛𝑒𝑥𝑡
¿ ∑ 𝛿𝑛𝑒𝑥𝑡∙𝑤𝑘𝑘𝑖 ∙ 𝑜′ 𝑖 ∙ 𝑜 𝑗(𝑘¿ 𝛿𝑖 ∙ 𝑜 𝑗)Sample Problem(1)Hand Literature

XNOR Gate

Sample Problem(2)Accuracy is 30
to 50%

There is no correlation

Introduction (2)Motivation

・ Analyze map of eye-gaze moving and obtainkey parameters to characterize human visual search

FOVE 0 VR

・ Propose our original mathematical analysis
methodologies
・ How to get hints of VR Sickness ?

Methodology
・ Obtain and analysis eye-gaze tracking data on
FOVE 0 VR system using original software
・ Clustered by K-Means method and Elbow method
・ Classify using Gaze Density and Reaction Time・ Fourier Translation and Neural Network Technique
for transient analysis

Eye-gaze tracking system