The application of emotion and mind
recognition in voice
Quantitative measurement technologyof emotion
The University of Tokyo
The Department of Technology
The Department of Medicine

AGI Inc.＋ PST Inc.
PhD. eng. Shunji MITSUYOSHI,
16.3.2012www.agi-web.co.jp

● Just simple● Beyond your gender, age or nationality● To be analyzed your emotions and mind
● Scientific real-time measurement
Anger

Anxiety

Excitement

Joy

ReliefCalmness

Sadness

Pain

Visualized emotions andits strength in colors

Dis/pleasure
2

ST/PST Review
ST = Sensibility Technology
VEA = Voice Emotion Analysis
PST = Psychoanalysis System Technology

Visualized emotions （ST/VEA）
Recognized emotions from voice (VEA) soon after utterance （for ９ emotions）
Shows even complex emotions easy
displayed in colors and degrees (volume/strength)
To know feelings ：Analyzed emotions in colors and shows its quantities

ST Emotion SDK

Recognized human’s mind （PST）
To know a condition ：Stresses, Fatigue
To know physiology ：Brain, Nerve, Homeostasis of hormone
The aspect of a cranial nerve transfer substance is analyzed from voice.
To know mind ：Determined manic-depressive psychosis
Supports ：The complement of a cognitive function to a disabled person
Improves ：Application to cognitive behavior therapy
STCRM Mind analyzer
3Made in Japan is…
Japanese people search for
perfection from the beginning.
• In these national traits, we produced highly efficienttechnology in Japan.
• We tried hard for 15 years.
• GAME, CRM, mental health, Smart Phone etc…
• USA, China, etc…

About our technology
Our technology has been recognized as formal adoption
by the world first army in advanced nations, doctor's
degree acquisition, the international patent, the National
Research Institute test result and the test result in the
Ministry of Defense (Japan Army).
The effect in the East Japan great earthquake wasinspected.
Thereby, it has proved that technology of our company
is the highest performance.
The performance was checked using (the huge subjectivity assessment data + brain measurement + blood test + doctor).

5

ST(VEA) Archive2001 ez virtual talk / KDDI
2003 Android SAYA / Tokyo University of Science
2005 感性生命体んとと君 / NTT lnstitute2007 Nintendo DS Kokoro scan / SEGA
2009 Sensibility Call Center / NTT soruko
2010 意識空間 / Mitsui Sumitomo construction Inc.
2011 The Defense Agency, National Defense Academy,
National defense medical school / A supporting teamfor the great Japan earthquake

Stanford
Released thetheory of ST thatmeasuresemotions

Started toobserveemotions

Inspected
correlation between
brain and emotions

2001 ST engine

2004 ST engine Ver.2

AGI Holdings
▲
2000

▲
2001

▲
2002

Completed the test
for forecasting
behavior at major
call centers

DSR

Verified detection of thedepression
2008 ST engine Ver.3
AGI Incorporation

A subsidiary company of SGI Japan Inc. （AGI）
▲
2003

▲
2004

▲
2005

▲
2006

▲
2007

▲
2008

▲
2009

▲
2010

▲
2011

携帯電話

ST 音声Server

ST Call centre▲
2012★Applied ST as a stressdetector to use for thegreat Japan earthquake
support in the selfdefense forces*No.1 official advanced
technology in military6

Achievements in 2000 to 2004
Verbal communication system, communicator

学生

ST Learning Server

ST Learning （ver.1.0）
日本大学
Innovation for language
materials

Virtual robot 「んとと」By NTT Cyber solution
laboratory Institute
Trial with “Untoto-kun” forseeking a new interface tohave emotion
携帯電話

ST 音声Server

ST artificial intelligence system「Setsugu(Service)」 （ ver.1.0，2.0 ）The virtual agent invoice recognition and emotion
analysis「EZ Virtual Talk」 KDDI Virtual character「Star Private Talk」 Virtual talk with public figure（ ver.1.0 ）7Achievement in 2005：Communication Support
Terminal

Designed by NEC

The feeling communicator

Express your feelings withlighting「言花 （KOTOHANA） by ＮＥＣ」
Exhibited 「CeBIT2006」■Date/Place・March 9th – March 15th, 2006 (7 days)・International Exhibition Center in Hannover,Germany
・6,270 companies were participated・Visitors: 48 millions （statistics by 2005）

Practical example

8Video conference system plus 言花 KOTOHANACebit 2006 KOTOHANA NEC

Achievement in 2007：NINTENDO DS•ココロスキャン

Released by SEGA
Total product sales
reached 70,000 sets
The children of
deafness said to me.
“ For us, mother’semotion came to be insight by a color the firsttime with your ST ”

Analyzed emotion in voice
and displays as a color
Applied ST into NINTENDO DS

10What is emotion?
Could we touch emotion?
It is subjective. It is sensibility.
How does it measure as subjective feeling and emotion?
Perhaps, is it similar to color recognition ?
If so, how does it explain about on the field of color research ?
We are going to explain emotion which is observed fromsubjection and physical function.

11

Outline of standardization for emotion andpsychology
1. Updated result of research on emotional expressions
2. Determined items of emotion – included synonyms
3. Updated physiological reaction and its matrix with emotion items （Matrix ofcorrelation between physical action and biological materials)
4. Built mechanism for a correlation of each items
5. Standardized emotion items

Emotion labels
Emotion’s matrix

Standardization

Emotion dictionary

Emotion’s mechanism
12

Emotional RGBYGREEN

How many emotions
are there?
Listed up 4500 ofemotional or
related words
from dictionaries

YELLOW
Narrowed
Grouping
down
into 223 labels 4 colors

RED
BLUE

By MITSUYOSHI (1999~2006)
13

223 words of emotion labels
This is a chart which is classified 4500 emotional related words into 223 groups

References : The psychology dictionary, The encyclopedia of 広辞苑 ・The great
Japanese dictionary , Oxford English dictionary, The medical psychoanalysis
dictionary and others
By Mitsuyoshi (1999~2006)

14

The research of relation between Emotional response / Physical reaction and materials
物質

興奮

ストレス

不安

CRH

覚醒

○ACTH
○

◎

神経ペプチド

沈静

○CRH
◎

○×

コルチゾール○

パゾプレシン

○恒常性

ACTH
前◎

CCK-8

全◎
前×

メラトニン

◎

CRH◎

アドレナリン

◎

期待

心拍

○

◎

瞳孔

交感神経

発汗

体温

活性

血圧

周期

◎

日周

◎

日周

○

○

◎

CRH◎

CRH◎

ガラニン

CRH◎

ソトマスタチン

CRH×

NA×

◎

◎

◎

◎

J◎

◎
◎×

低◎

CRH×

GABA

CRH×

NA×

×

◎

◎

×

忘却機能

○

日周季節

NK活性

運動快感

◎

◎

拡大

緊張記憶

◎

拡大

緊張

記憶

◎

×
NA×

ジアゼパム

NA×

NA×

NA×

NA×

エンケファミン
βカルボリン○◎

イソプレナリン

◎

ヨヒンビン

◎

フェンフラミン◎

乳酸ナトリウム

◎

CO２

◎強
◎

◎強

感情や精神状態（横軸）、および身体反応（縦軸）に影響するという確認が学会誌や論文などの治験にある、生体物質やホルモンなどを調べ、対批表マトリックスにした。
目的：脳の活動と伝達物質・ホルモンから導かれる情動の発生メカニズムの推定○○は合成/分泌，○×は分泌して抑制，◎は促進，CRH ◎はCRH 合成・分泌促進，前◎は前頭葉皮質で促進，全◎は脳全体で促進，前×は前頭葉皮質で抑制，低◎は低下したら促進，×は抑制，○恒常は分泌により恒常性を維持，CRH ×は，CRH
合成・分泌抑制，◎×で制御，△は調整，NK はナチュラルキラー細胞，
活性は免疫機能の活性，機能は免疫機能，日周はバイオリズム，?は報告情報，- は文献[9] に記載なし
表を見てわかるように- が多く，心と感情と分泌物質の関係では不明な部分が多い．CCK 系統と5-HT, GABA, DA は複
雑にお互い影響しあって情動に作用するようである．また，性ホルモンは闘争と深く結びついているようである．これらの分泌物や物質は脳神経活動に制御され，大脳辺縁系と情動と記憶は密接に連携している

15

×？
×

活性

表１：心的身体的作用と生体物質の関係マトリックス

エタノール

覚醒

◎

低◎

×

αーMSH

ガラニン

快感

○

◎

CRH◎
△

CRH◎

×

免疫

×
○ ◎強

◎

◎

アンジオテシン

オキトシン

陶酔

◎

幸福

CRH◎
△

セロトニン

ベンゾジアゼピ
ン

安定

◎×

NA×

沈静

◎

快不快

◎

○×

アセコルリン

カフェイン

うつ

○

CRH×

ドーパミン

恐怖

NA×

○ ◎強

βエンドルフィン

ノルアドレナリ
ン

闘争
◎

○

CCK-4

オピオイドペプ
チド

嫌悪

×？

母性行動

Presumption of Emotion
Model

Objective：
To be visualized correlation
between brain, emotion andaction based on medical researchProgress：
How built a map that shows amatrix what is comparable
correlation between mental or
physical operations andbiological materials

Modeled/Framed：
Represented substances andbrain function on a circular
diagram as indicated on the angle

仮説図（博士論文・日本機械学会編集「感覚・感情とロボット」より）

16

Four colors grouping (RGBY) of
２２３ emotionsi.

Composed the emotion
map classified in five colorsii.

The primitive reaction ofemotion from survival andbreeding is shown in centeriii.

Action shown in outline

iv.

To analyze emotion, read amap outline toward centerv.

To predict or analyze
behavior, read core area
toward outlinevi.

The outline is circulated sothat can be distinguishedindividual character. The
idea is enable to apply
psychiatric test in medical
field.

“Positive” emotion

NeutralemotionPaleemotionPaleemotion“Negative” emotion

仮説図（博士論文・日本機械学会編集「感覚・感情 ロボット」より）
17

Excitement (activation) andAnxiety (corruption) Emotions
The advanced study of emotions’ mechanism

Activation
Corruption

The core area thatshown 18emotional
feelings（A color chart included
dis/pleasure
movements）

Hypothesis by Dr. MITSUYOSHI

The position of
excitement andanxietyemotions

Anxiety (corruption) may happen
upon the condition above18

Body structure A<A short term effect on brain emotion activity>
Human has a strong emotion, it is usually under control ofintelligence. This is the human emotion.
Usually human modulate emotions well on social life.

EmotionA vocal tract is sensibility affected
what to restrain emotions = Modulated

limbiccortexhypothalamus
and the midbrain

Voice generation causes function
with voluntary muscle

Filtering（Formant）

Speaker

ST analyze
a brainfrom
vocal cords
and analyze
emotions.

Involuntary

The
Lungs

vocal cords

The voice movement like freezing, trembling or
stammering voice are a sign that involuntary
affected by brain emotional activity

In a vocal cord , the involuntary
emotions reaction which occurredin "limbic cortex, hypothalamus
and the midbrain“ comes out.

The vocal
cords

Since this portion is a place
which generates language
intentionally,a lie attaches it in this portion.

The vocal
track

Mouth
19

Body structure B<Feedback＝a long term effect of feelings (emotions)>

limbiccortexhypothalamus
and the midbrain

A core of emotion affect in short. In general, when a strong emotion
appears in a brain, it occurs high blood pressure by a hormones or afrequent pulse.
Focusing physical action may be concluded that it is a long term
effect of emotion.
Beside, a hormone effect goes back to a brain and thus it is difficult
physiologic measurement considering a recognition effect.
Only voice output has not fed back in the circuit of human body structure.
Therefore, the sound can carry out the output of the related state of a brain
and the body directly..
Usually both link with each other.
So, it become blur what is happing

Fed back!

Important

Hormones

It is an involuntary activity, and feed backComplexity system
Actually it is affected by blood circulation andbiological materials

20

VEA structure / The concept ofemotion measurement①Controlled intelligence＋
affected feelings＝subjective
element

Control of theemotions byreasonEmotionThe logic structure derived
from emotional awareness.
Vocal
tract

Under controlled by
prefrontal area
Under controlled by
Cerebral limbic

OutputVocal cords
A Vibration
source②Emotional brain
action/the nerves
effectWe derive soundparameters,
immediately andinvoluntarily from thevocal chord

VEA subjective
feeling analysis
Logic comparative chart

Feelings
measurement

Emotionmeasurement

Emotional
feelings

VEA nervous systemanalysis
Voice parameter

Organs・Body⇔
Nerves・brain aspect
21

VEA structure and its output
Emotions and respiration are both related.
VEA is enable to analyze emotions at utterance interval between a breath to a breath
Speaker

The ST VEA Internal structure
Subjective
analysis

EmotionEmotions

Anger degree
Joy degree
Sadness degree
Calm degree
Excitement
degreeDis/pleasuredegreePain degree
Anxiety degree
Relief degree

Sound parameters that
related to emotions（wavelength quality etc.)

Reproduced feeling (subjective)
pattern(parameter threshold setting)

The strongest emotion will be selected by majority decision on judgment logic
and display as classified colors and strength.
An other emotions will also list following the first chosen.
22Fundamental frequency detectionof solid vocal cords
A voice wave

Previous Cepstrum
presumption
shows a sparse wave.
Not to be automatic
detection

Measurement failed : Appear sparse spots on a monitor

It made possible toanalyze subjective
emotionautomatically
VEA presumption
method applied nonNo sparse wave
Cepstrum enabledpresume a basic
wavelength underambient noise
光吉俊二 博士論文「感情認識及び情動の脳生理信号分析システムに関する研究」から

23

F0 ST ver2.0

ST Software
limbiccortex(Principle)
The result of
havingcontrolled
emotions

hypothalamus
and the midbrain

Speaker

Emotional
brain action(Autonomic Nerve)

The state of an emotions element
and subjectivity control
componential analysis area

its balance

The activator
action state of
the brainemotions of anerve level(Real intention)

The highest result among analysis
outputSadness, Pleasure, Anger, Calmness

Power of emotions = Excitement

Real-time procedure （A time axis）

ST enabled to visualize quantity of emotions in colors
特許取得済みST2004

Evaluation of human's
feeling utteranceStandardized subjective feeling <Learning data （2800 speakers) Test data （1100 speakers）>

Natural emotional utterance data:
Conversation was recorded showing the image out of which
natural feeling tends to come.

Acting emotional utterance data:
The utterance person did feeling utterance using the scenario.

Evaluation technique：
100 evaluators (man-and-woman age was equalized) judged
emotion for "the sound put together at random" repeatedly with"the exclusive tool which judges a sound unconsciously."

voice samples labeling with over 100 persons of subjective views
50,000 of utterance （2800 examinees）×100 persons of subjects
1 hour / person Work of 300 utterance evaluation = 11 years survey
27

Subjectivity reappearance mechanism
Subjectivity learning data

Analysis parameter

Analysis result

Subjectivity analysis
Subjectivity reappearance logic

etc

Study data (2800 of speakers ）stored with 100 subject views

analyzed with 200 or
more fixed parameters.

A fruit is very serious.
An ordinary researcher
becomes insane!

All the members find out
Subject is replaced with
common subjectivity separation
judgment logic rule
regularity.

Study data
Self evaluation

A

Others evaluation

A∩B

B

Although various analytical skills (HMM/NN/SVM/…) were failed, we reached tothe practical use level by manual labor and rule base analysis.

28

Evaluation of ST(VEA) (Voice Emotional Analysis)
ST(VEA) system (Open test)
Subjective analysis test data
1100 speakers

Reproduction of subjective analysis
Parameter and logic-rule set（Not used learning data）
If it divide using a color
like people's subjectivity,
it can be said that the logic
has reproduced
people's subjectivity.

etc

Sets of emotional voice
data labeledin objective andsubjective analysis

Unsolved: Is it can be separated as same as subjective feeling in solidparameter and judgment logic ? Testing data required.

system right after utterance

Test data
Self evaluation

A

Others evaluation

A∩B

A：The data that evaluates emotion by oneself using the VEA

B

B：The data that evaluates speaker’s emotion judgment objectively using
VEA system as above A .

A∩B ：Set the voice data which was corresponded to evaluation (about
1/100) as study and test data.

29

The subjectivity reappearance
performance test of ST(VEA Open test)

IEEE2007NLP-KEで発表

Reproduction of subjectivity
A : Joy & Calmness
B : Joy & Anger
C : Anger & Calmness
D : Anger & Sorrow
E : Sorrow & Calmness
1 Point = 1 word
Emotional elements are segregated inparameter space.

A vertical axis and a horizontal axis are various parameter sets.
B is the composition of the set which separates anger and joy.

ＩＥＥＥ:「Emotion Voice Analysis System Connected to the Human Brain」 Shunji MITSUYOSHI

30

Evaluation of VEA by human listeners
Free conversation between speaker 1 and 2

Speaker1
Self report

Speaker2

Listener1,2
Match
vs

VEA 1

Subjective Judgment

etc

VEA2

VEA observed the emotional elements of speakers 1 and 2
independently

31

The subjectivity coincidence rateof ST(VEA) Intuition test

Recognition capability of feeling, and comparative experiments of the recognition rate of ST

100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%

本人主張 speaker(日本人・正解)
vs
Japanesevs
聞き手 (日本語を知らない外国
Listener
subjective (Non人・比較)Japanese)
本人主張 (日本人・正解) vs

Evaluation limit(Speaker’s subjectivity limit)

聞き手 (日本人・比較)Japanese speaker vsListener
本人主張subjective(日本人・正解) vsST (機械・比較）
Speaker (Japanese)

Speaker

ST

ST

86%

70%

Japanese
本人評価 speaker vs ST(Independent of language)

Japanese

Speaker's subjectivity

Japanese
NonJapanese

NonJapanese

感情

Sorrow, Joy, Anger, Calmness

情動

Excitement

Automatic voice emotion recognition outperformed human‘s subjectivity.
As for the excitement, the non-Japanese and the Japanese are equal to VEA.
ＩＥＥＥ:「Emotion Voice Analysis System Connected to the Human Brain」 Shunji MITSUYOSHI

32

Comparison of ST(VEA) andbrain emotions activity

33

fMRI and physiology system
We constructed a system thatparameterizes the brain activity andphysiological body state.

Stablemeasureetc

The advantage:
Stable physiological measure makes
influence of subjective feelings
minimal.
Problem:
MRI sound noise is extreme (130dB).
Difficult to conduct sound experiments.
By NICT + AGI (2005-2008)
National Institute of Information and Communications Technology (NICT)34

Solution: A noise-free microphone and ahead set system createdSpongeMain body of hazardous mask
Heavy rubber seat
Dustproof and Fsedoc filter

Paper filter Rubber mule + silicon rubber in the mouth.

The F0 detection and the VEA were achieved under the fMRI noise of 130 dB
By NICT + AGI (2005-2008)
National Institute of Information and Communications Technology (NICT)35

MR noise reduction by 80 dB
with a filtered-microphone

A non-filtered-microphone-80dB

A filtered-microphone
By NICT + AGI (2005-2008)
National Institute of Information and Communications Technology (NICT)36

Procedure and analysis☆MRI: Trio (3T, Siemens, Germany):
BOLD was measured for the subject whowore a filtered-microphone with headset.
Subject had conversation with theexperimenter for 150 x 2 seconds or 300x3
seconds. 1 min rest followed theconversation. The director guided
conversation if little sign of emotion.
Experimenter: YT, Subject: SM, TO, TN.
Director:
Koichi Shibasaki (AGI)★Emotional Analysis:
Emotion (excitement), feelings (normal,
anger, fear, laughter) was identified by
the VEA system (ST2.0TM ) producing
the event trigger.
☆BOLD analysis (brain):
SPM2.0 (UCL, England) was used on
MATLAB v6.5.
★Event-related analysis done for each emotion
event and compared with the rest of theperiod.
★Informed consent was obtained fromall subjects.

By NICT + AGI (2005-2008)
National Institute of Information and Communications Technology (NICT)
37

This is a proof with which subjects talkand sympathize.
BOLD Activation• Motor cortex＜A motion of a mouth＞• Blocca area
＜Conversation utterance＞• Welnicke area
＜Language recognition＞• Right Blocca area
＜Sympathy, mirror neuron＞

Feeling is induced byconversation.

R

L

R

L

L

R
By NICT + AGI (2005-2008)38

Emotional judgment by ST(VEA)
indicating negative emotion

Judgment by VEA

Recognized as “excited” and “anger”

No emotion recognized
We compared the brain activities when recognized as“excited” and “anger” (triangle)
with that when no emotional elements are recognized.
Statistical analysis was conducted by one way t-test.

The theme of medical studies has only "research of negative emotion."
Time (seconds)

By NICT + AGI (2005-2008)

National Institute of Information and Communications Technology (NICT)

39

This is a proof which ST had judged to be anegative emotion only when brain activity was anegative emotion.

Activation area:(t-test, p<0.001,)

R

1．Left and right frontal BA44,45＜L:speech generation, R:
sympathy ＞
2．Left dorsofrontal BA12
＜hormonal control, suppressive
emotion＞
3．Left amygdalate complex＜negative emotion＞
4．L/R inferotemporal＜visual imagery, body image ＞

Why visual area?
S’s imagined experimenters face.

L

L

R

R

L

R

L

L

R

By NICT + AGI (2005-2008)
National Institute of Information and Communications Technology (NICT)
40

２００９
Action prediction test
We performed the behavioral prediction experiment.
It has done to compare with voice data shown by ST(VEA)
and the fact of action <payment, nonpayment, suicide andadmission> at a major domestic credit company.

The experiment has done used 2000 voice files selected at random.
We produced <a software which predicts customer's “attribute and behavior"> using ST(VEA).
The fact of customer's action was verified later. It asked for the coincidence rate.
Now, precise ratio of prediction is 80 %. (reached 90 % after a week)

Under the condition of non-payment rate 50%.

41

2009
Voice analysis research of"the depression andthe anxiety disorder state"
for the prevention from suicide
42

The problem of depression diagnosis
Electrocardiogram

NIRS
ICD-10
Genuineness"depression"?

fMRI
The true value of"depression"?

SCID

This is cloudy!

Blood test
BDNF

DSM-4

In the present condition, the clear index of "depressed state" have not be appeared.
43

There is no technology
measured every day!
We have faced issue that
nobody is able to quantify
depression until now!
"Depression"
MeasurementDiagnosisStatemaintenanceMedical aid

Feedback
Observation

An old consultation questionnaire
1: "Depression" is normally not identified causes that patients
often tries to hide "depression."
2: Lie Attaches.
ex. Those who want to neglect to work pretend as if they have"depression."

A social
contribution
increases
rapidly.
44

PST enables medical innovation.

A brain is analyzed from a voice.
PST allows diagnose to identify illness easily.
PST = Psychoanalysis System Technology

扁桃体

視床下部
中脳

Cranial nerve
voice parameter
Biological material
voice parameterST
45

A possibility of PST is speech analysis of"the five states from a cause for a disease"
It seems that this area
has strong influence ofa feeling.
Then, the subjectivity analysis
logic rule set of ST analyzes.

Opposite-sex
problem
Work/housework
child-care

The cause ofenvironmental
statusMonetary matter

The cause ofcognition nature

Dementia

Weather condition
Culturel trait

Long - term measurement

A psychogenic
cause

FamilyproblemStressBody state
Body disease

A nervous cause

Neuropathy
Brain disease

Diseases of
the nervous
systemIt seems that this area
has strong nerve bodyinfluence.
Then, it analyzes withST nerve soundparameter.

Momentmeasurement

The cause ofphysicality

46

PST2009
Psychoanalysis output
Manic - Depressive
Personality disorder
Manic Parameter

DoctorSpeaker A
Input line
patient
Speaker B
Input line

Autonomic nerve effect(Excitement parameter)

depression

Parameter47

PST2009

PST2012＜ｆＮＩＲＳ／ｆＭＲＩ／ＰＥＴ/EEG….＞ＶＴＲ

Blood test

DB

Nerve activity measuring device

49

Living body secretion measuring device

PST2012

PST ver1.0

特許出願中

50
特許取得済みApplication: Medical test
Are they individual difference or a pathology difference?
* Neurological analysis（NIRS・fMRI） comparison of tested
Voice and healthy (control) voice.
* Interview (SCID): comparison of tested
Voice and healthy (control) voice.
* DSM-4/ICD-10: comparison of tested
Voice and healthy (control) voice.
* Heart beat: comparison of tested
LF/HF?
Voice and healthy (control) voice.
* Blood test: comparison of BSDN tested
Voice and healthy (control) voice.

Neural parameter
Logic setComparison
With Doctor
Diagnosis*Mental comparison* Neuronal diagnosis

Synthesis
Diagnosis
Comparison
Adjustment

show

51

52
特許出願中

特許取得済み特許出願中：2010-113446

53
特許出願中

特許取得済みST Emotion (VEA)
Software Development Kit● Required platform－ Windows: Windows XP, Windows Vista（developing environment: Visual C++ .net 2003 and later versions）● Sound format－ PCM WAVE format● Required hardware－ CPU: Intel Celeron processor，or compatible processors:
speed more than 1GHz
－ Linux: Fedora Core 5/6, open SUSE 10.0 or later versions－ Sound boad: sampling frequency 11kHz, 16bits or more.
－ Memory: 512MB or more （recommended）
54

The stress research
The basic principle of ST/PST
Proof in which the National Defense Medical
College did formal adoption

The Ministry of Defense andNational Defense Medical College joint research
Ministry of Defense formal adoption
was carried out.
For the first time in the world, formal
adoption of the voice psychologicalanalysis system was carried out inthe defense organization ofadvanced nations, and science
proof of the principle was carried out.
as stress detector

The Self-Defense Forces
Subject group
Army school student
Special force member

Normal group
Stress group

To adopting PST, the Ministry of
Defense studied performance ofdomestic and foreign devices
compared with PST .
As a result, PST showed significant
performance and capabilities.
PST has been proved as stressdetector .

Overseas dispatch member
East Japan great earthquake
contingent member

Prolonged stress group
Severe stress group55

Application of the stress resultof research
The Ministry of Defense and National Defense Medical College joint research Practical use
by the East Japan great earthquake
We measured "the Self-Defense official's stress" in the East Japan earthquake disaster.
World defense Science Research Council (DSR) announced

Result: Comparison of PST to a stress index

He won the highest prize in
an international force
medical congress.

Coincidence with a blood test (herpes bacillus trace)
Compatibility with a doctor's diagnostic result
Coincidence with a subsequent condition change
World defense Science Research Council (DSR)
International force medical congress (award)
Medical Tribune (printing)
JSTSS, Japanese Society for Traumatic Stress Studies(Refereeing passage)Now, the contribution to Nature is prepared.

56

Quality assessment conclusionof ST/PST
Scientific verification
Subjectivity and ST：To compare human’s subjectivity with ST
A: ST enable to reproduce emotion as same as human being.
B: It has been proved that ST performs accurate evaluation of what speakers actually felt.
Also accuracy for this evaluation is higher than what outsiders presume it.

Brain activity and ST：To compare brain (excitement) with ST used fMRI
C: When the brain activity showed negative reaction with the rate of disagreement 0.1%,
ST also judged feelings as negative （by six examinees using fMRI（3T）device）

Stress and PST：Stress index and PST with Blood test + doctor’s result + condition
D: Coincidence with a blood test (herpes bacillus trace)
E: Compatibility with a doctor's diagnostic result
F: Coincidence with a subsequent condition change

Practical examples
Action and ST：The behavioral prediction experiment at a major domestic credit company
ST device succeeded to predict consumer's action with 80% accuracy without voice
analysis under the condition of non-payment rate 50%.
(Reached 90% accuracy after a week）
Our technology has been recognized as formal adoption by the world first army in advanced nations, doctor's degree acquisition, the international patent, the National
Research Institute test result and the test result in the Ministry of Defense. The effect in the East Japan great earthquake was inspected.
Thereby, it has proved that technology of our company is the highest performance.

57

Standardization of emotional RGBY ver-1.0

Regulation

Activation = EX
Corruption = CO

1stcircle2nd
circle3rd
circle4th
5th
circle

6th

7th

8th

Basic emotions①

STY <P> -L -L②

STR <N>-D -Ag③④

STB <N>-U -Ax
STG <P> -S⑤

STR B <N>- 3 7 (EX89)(CO=and STB<N>-D-P

n
i=1

Xi)

red-violet ⇔ blue-violet ？

Circulation

X= Continuous quantity= Quantity division chain operator
a b = Change thresholdOur demand•

The partner who can research together withPST overseas.
1. Clinical study (Brain, Nerve, Intervention trial)
2. A little different from one culture to another (UI)
3. University PST laboratory is start•

The partner who gets business together withus overseas.
1. With the existing technology (ST = civilian goods)
2. With the newest technology (PST = med-tech)

ST is AGI.Inc.
PST is PST.Inc.

Thank you!
information @ agi-web.co.jp