SUGISAKAMASANORI

Journal of Advances in Artificial Life Robotics
Vol. 1(3); December (2020), pp. 153–157
ON LINE ISSN 2435-8061; ISSN-L 2435-8061
https://alife-robotics.org/jallr.html

Research Article

Basic Study on Motion Control ofElectroencephalography (EEG) analysis

CG

CharactersbyMakoto Sakamoto1, Kenji Sakoma1, Takahiro Ishizu1, Takahiro Shinoda1, Amane Takei1, Takao Ito2
Department of Computer Science and System Engineering, University of Miyazaki, Miyazaki-City, 889-2192, Japan
Graduate School of Engineering, Hiroshima University, Higashi-Hiroshima, 739-8527, Japan

1
2

ARTICLE INFO
Article History
Received 10 November 2019
Accepted 03 July 2020
Keywords
High-power joint
Link mechanism
Cam mechanism
Design principle

ABSTRACT

Virtual Reality (VR) technology is expected to develop in various fields such as medical,
education, business and entertainment. In this study, we aim at more intuitive operation byfocusing on troublesome mounting in VR. To achieve this goal, we propose manipulation methodof CG character by electroencephalography (EEG) with relatively inexpensive equipment. In thispaper, we ask five subjects to manipulate CG characters by EEG and evaluate this system byquestionnaire.

© 2022 The Author. Published by Sugisaka Masanori at ALife Robotics Corporation Ltd.
This is an open access article distributed under the CC BY-NC 4.0 license(http://creativecommons.org/licenses/by-nc/4.0/).

1. Introduction

to select a place when using a VR device. Recently,
integrated (stand-alone) models that do not require any
cables are gradually appearing. The integral type solves
the problem of cables. However, space problems aroundthe user have not been solved [1].
Therefore, as a method for solving this problem, it is
proposed to use an operation by electroencephalogram,that is, a method called BMI (Brain Machine Interface).
BMI, also called BCI (Brain Computer Interface: Brain
Computer Interface), is a study that measures brain
waves with an instrument, extracts features from thebrain waves, and causes the computer to react in some
way according to the features. We think that space
problems around the user may be solved by using thismethod [2], [3].
In this paper, we aim to manipulate virtual objects moreintuitively by allowing a user to perform basic
manipulations, such as creating virtual objects,
deforming, moving and combining them.
We make it possible to realize the interaction of the user
and the virtual objects by adopting the Leap Motion

Since 2016, the so-called the VR (Virtual Reality) firstyear, VR technology has made remarkable progressaround the world. Every year, higher-level VR devices
are released, and although the price is somewhat high, it
has become easy to purchase for us. As a result, VR
technology is expected to develop in various fields suchas medical care, education, business, and entertainment.
However, there are five major problems that must be
solved in the development of VR technology. The firstone is the problem of troublesomeness of wearing, thesecond the problem of specifications, the third the socialand market problem, the fourth the problem at the
production site, and the last the health problem. Amongthese, I focused on the first problem of troublesomenessof wearing. When using VR devices, it is necessary to set
up many cables and sensors. Also, there must be a certain
space around the user when using it. This is for theconsideration of the surroundings or the safety of the user
extent when performing any operation in the VR world.
For this reason, there arises a problem that it is necessary

Corresponding author’s E-mail: sakamoto@cs.miyazaki-u.ac.jp, itotakao@hiroshima-u.ac.jp

153

Used to receive brainwave data from Mind Wave
Mobile in Visual Studio 2017.

Controller specialized for acquisition of threedimensional (3D) coordinates of fingers and introducing
flexible body virtual objects using Bullet Physics. We
also perform a hidden surface removal to allow the user
to recognize the context of the user's hand and the virtual
objects.

MathNet.Numerics
2) Accord
3) Accord.MachinerLearning4) Accord.Math5) Accord.Statistics

2. System Component

Used to process data received from Mind Wave Mobile
in Visual Studio 2017.

In this study, it is necessary to measure brain waves.
Therefore, the above equipment was used in this research.

2.4. Brainwave measurement

Generally, the BMI research uses an international 10-20
electroencephalograph, but in this research, theequipment sold by Neurosky Inc. from the viewpoint of
whether BMI can be realized inexpensively (see Fig. 1)
was used.

In this study, it is necessary to measure brain waves. In
the general BMI research, the international 10-20 method
is used, but in this research the brain wave measuringmachine Mind Wave Mobile manufactured by NeuroskyInc., which is relatively inexpensive. because it is alsothat it is from the viewpoint of verifying whether BMI ispossible even with a simple device. When receiving EEG
data in Visual Studio 2017, I used a library called
libStreamSDK.

2.1. Used equipment
1) Mind Wave Mobile.

In addition, when acquiring an electroencephalogram,the kind of electroencephalogram acquires a Row wave
which is not processed at all. Then, extract features in
Visual Studio 2017.
2.4.1. FFT (Fast Fourier Transform)
It is a kind of DFT (Discrete Fourier Transform), and isan algorithm designed to execute DFT fast on a computer.
When using this analysis method, I used the library of
MathNet.Numerics [5].

Fig. 1: Mind Wave Mbile.
2.4.2. Continuous wavelet transform
2.2. Development environment
The development environment for constructing thissystem is as shown in Table 1 below.
OS

Windows 10

Programming
languageC#

Measuringequipment

Mind Wave Mobile

Software

It is possible to use a wavelet function as a basis
function and leave temporal information that is lost in the
Fourier transform. This is a method that attempts torepresent the waveform of a given input by scaling,translating, and adding wavelet functions (see Equation
1).

𝑊𝑊𝑊𝑊(𝑎𝑎, 𝑏𝑏) =Visual Studio 2017
Unity 2017.3.1f1

1√𝑎𝑎� 𝑓𝑓(𝑡𝑡)𝜓𝜓 �

𝑡𝑡 − 𝑏𝑏
� d𝑡𝑡
𝑎𝑎

2.4.3. Discreate wavelet transform

Table１: Development environment．

(1)

The original signal is decomposed into the highfrequency component and the low frequency component,
and the decomposed low frequency component is also
decomposed into the high frequency component and thelow frequency component (see Equation 2)

2.3. Library1) libStreamSDK

154

𝑑𝑑𝑘𝑘 = 2𝑗𝑗 � 𝑓𝑓(𝑡𝑡) ��������������
𝜓𝜓(2𝚥𝚥 𝑡𝑡 − 𝑘𝑘) d𝑡𝑡
(𝑗𝑗)

𝑅𝑅

2.4.4. EEG feature discrimination method(2)

Generally, As EEG feature discrimination method,
SVM (Support Vector Machine: support vector machine)
generally used in EEG feature discrimination was used.
SVM is one of pattern recognition models using
supervised learning (see Equations 3, 4 and 5). SVM has
high discrimination performance against unlearned data.
In this study, Gaussian kernel was used as the kernel
function. In addition, the library of MathNet.Numerics
Accord, Accord. MachinerLearnin, Accord.Math, andAccord.Statistics was used to execute SVM [4].
y(x) = sgn � � wn K(xn , x) + b�

=�

n=SV

0 𝑥𝑥 ∈ 𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐
1 𝑥𝑥 ∈ 𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐

Fig. 2: The CG character which used in Unity．
3. Experimental method *The subject sits in a chair and starts measuring after
wearing Mind Wave Mobile on the head. The brain wave
processing method uses FFT. During the experiment,
subjects are asked to relax moderately or move their body,and so on, to evaluate how CG characters in Unity move.
In this research, CG characters move only in two types,
and they are stopped when relaxing and move forward
when trying to move their bodies.

(3)

(4)

K(x1 , x2 ) = exp(−‖x1 − x2 ‖2 ∕ 2σ2 ) (5)2.4.5. Visual Studio 2017

4. Evaluation experiment

Visual Studio 2017 processes brain wave data sent from
Mind Wave Mobile and performs feature detection. At
this time, the brain wave processing method used FFT.
Thereafter, the processed electroencephalogram data issubjected to SVM, and the electroencephalogram isdiscriminated such that 0 is output when relaxing, and 1
is output when user tries to move (see Equation 4).

In this experiment, five university students became
subjects, and they conducted evaluation experiments andasked them to answer questionnaires. The contents of theevaluation were in the form of free-form descriptions of"good points", "bad points" and "others".
5. Evaluation results

2.4.6. Unity 2017.3.1f1

As a result of conducting the questionnaire, it became
as follows.
"Good things" were "Simply fun", "I was interested
because I wanted to move the body, so I was interesting","CG character was cute" and so on.
"The bad point" were "It took time for the CG characterto actually reflect the reaction after trying to relax / movethe body," "Mind Wave Mobile is coming off," "Mind
Wave Mobile was tightening my head and hurts me", "Itmoves even though it is relaxing. / The character freezes
while trying to move the body." and so on.
"Other" were that "is it not compatible with othermovements?" and "is there no other CG character?", etc.

We created CG characters and programs that change in
response to the features of the brain waves (see Fig.2). It
was processed in Visual Studio 2017, and it received thefeature which was judged by SVM, and made it torespond according to the feature. In addition, thischaracter uses the one promulgated free from Unity's
Asset Store.

*1556. Consideration

improvement of troublesomeness when wearing Mind
Wave Mobile, and diversity of motion of CG characters.

From the evaluation results, it was found from the “goodpoints” that the control of CG characters by EEG whichmeasured by inexpensively equipment, which is the
purpose of this research, can achieve certain results withregard to only two simple patterns of forward andbackward.
However, from the "bad point", it was found that themethod of detecting the characteristic of theelectroencephalogram and the discrimination method
were still incomplete. We also found that there was a
problem with the installation of Mind Wave Mobile. Thisis considered to be a problem that the size cannot bechanged or cannot be selected. From the above "badpoints", it was found that this research needs further
improvement.
According to the opinion of "other", it was found thatthe operability and key CG characters in this researchrequire diversity.

References
1. 2017 Advanced Content Technologies, etc. Distribution
Promotion Business Report，Part I, guidelines for using
content production technology such as VR，2018
2. Tomoki Hidaka，Yuki Shiraishi，Yukiko Kawai，Jiro
Okuda ， “Information
controlsystembyelectroencephalogram using a portable terminal” ，Information Processing Society of Japan，2014．
3. Uchida Yu, Kikuro Fujimura, Yoshihiro Maegaki, “A
Study on Numerical Processing of EEG Data for EEG
Analysis，Japan Society for Fuzzy Theory and Intelligent
Informatics，2014．
4. Takio Kurita，“Introduction to Support Vector Machine”
5. Jun Sakamoto ， Yasuhiro Mori ， Takayoshi Sekioka ，“Probability Analysis Method By Discreate Fast Fourier
Transform”， J．Struct．Constr，Eng．，AIJ ，No ，472，39−45，Jun，，1995．

Authors Introduction
Dr. Makoto Sakamoto

7. Conclusion

He received the Ph.D. degree incomputerscienceandsystems
engineeringfromYamaguchi
University. He is presently an associate
professor in the Faculty of Engineering,
University of Miyazaki. He is atheoretical computer scientist, and his
current main research interests areautomata theory, languages and computation. He is alsointerested in digital geometry, digital image processing,computer vision, computer graphics, virtual reality,
augmented reality, entertainment computing, complex
systems and so on.

In this research, we focused on the "trouble of wearing"which is a problem in the development of VR technology.
Above all, we focused on the problem of spatiality
around the user, and researched that it could be solved by
the following method. We have proposed a method in afield generally called BMI : the electroencephalogram
data is measured using an electroencephalograph, theelectroencephalogram data is received by Visual Studio
2017, the electroencephalogram data is appropriately
processed, and then the electroencephalogram features
are discriminated by SVM, and the discriminant values
sent to Unity 2017.3.1f1 and operates CG characters inUnity according to the determined value. In addition,although an electroencephalogram measurement
machine conventionally uses the international 10-20
method, in this research, Mind Wave Mobile
manufactured by NeuroSky was used because it verifies
that it could be substituted by an inexpensive device. The
result of the evaluation experiment shows that althoughthemanipulationofCG
characterbyelectroencephalogram which is the purpose of thisresearch can achieve a certain result, its accuracy is badand there is much room for improvement. Future issues
will include brain wave feature detection, improvementof brain wave feature discrimination accuracy,

Mr. Kenji Sakoma

156

He is a master student at Department ofComputer Science and System
Engineering, University of Miyazaki.
His current research interests arecomputergraphics
andimage
processing.

Dr. Takao Ito

Mr. Takahiro Ishizu

He is Professor of Management ofTechnology (MOT) in Graduate Schoolof Engineering at Hiroshima University.
He is serving concurrently as Professorof Harbin Institute of Technology(Weihai) China. He has publishednumerous papers in refereed journals
and proceedings, particularly in the area of management
science, and computer science. He has published more thaneight academic books including a book on Network
Organizations and Information (Japanese Edition). His
current research interests include automata theory,
artificial intelligence, systems control, quantitative
analysis of inter-firm relationships using graph theory, andengineering approach of organizational structures using
complex systems theory.

He is a master student at Department ofComputer Science and System
Engineering, University of Miyazaki.
His current research interests arecomputergraphics
andimage
processing.

Mr. Takahiro Shinoda
He is a master student at Department ofComputer
ScienceandSystemEngineering, University of Miyazaki.
His current research interests arecomputergraphics
andimage
processing.
Prof. Amane Takei

He is working as Associate Professor for
Department of Electrical and systems
Engineering, University of Miyazaki,
Japan. His research interest includes highperformancecomputingforcomputational
electromagnetism,
iterative methods for the solution ofsparselinearsystems,domaindecomposition methods for large-scale
problems. Prof. Takei is a member ofIEEE, an expert advisor of The Institute of Electronics,
Information and Communication Engineers (IEICE), a
delegate of the Kyushu branch of Institute of ElectricalEngineers of Japan (IEEJ), a director of Japan Society for
Simulation Technology (JSST).

157