• SCOPE•• IOR SCOT
• SDN
•••••

SCOPE
Step1:Step2:iD
29 305•• SDN•28MRI••••••••MRI5• SDN
• SDN
••5BA

• SCOPE•• IOR SCOT
• SDN
••••

•

n5nnn4nnnnnnn

nnHP

24 4[3]n5(nn) (5)

69%

90%

25%

78%

7%

13%

• SCOPE•• IOR SCOT
• SDN
•••••

Intelligent Operating Room (IOR)nOpen MRIBrain structure confirmation after brainshiftn Conform residual gliomannReal-time update navigation system
Visualization of position surgicalinstruments on intra-MR imagen It is also possible to acquire
the surgical lognn

Open MRI

Microscope image

Microscope, etc.
Navigation system
Visualization of intra-operative
information
provided by devices is possible

Smart Cyber Operating Theater

cW
12

R

O

19

29

X

84

296
747
ME
2014

1994

1458

1970

26

B

Xi

-Taia

W
C

Ity

I

it

ay

dat

Way

B-ianArr
Ri-tyIraA

A

SCOTsyrpy

ミドルウェアを用いた機器のネットワーク接続
神経機能検査装置

術中迅速
フローサイトメータ

生体情報モニタ

神経検査

プロバイダ

生体モニタ
プロバイダ

手術ナビゲーション

電気メス

FSM

プロバイダ

電気メス

プロバイダ

※画像のみ

無影灯

術野画像

手術ナビ

プロバイダ

通信プロトコルに依存しない

HDSDI

プロバイダ

DICOM画像
プロバイダ

無影灯

プロバイダ

機種・ベンダーに依存しない

Data Server
統合表示

時刻同期

データ保存

工程解析

意思決定

アプリケーション層

デバイス
監視

ロボット
制御

クラウド
接続

2015

2-•
•
•
•
•W
arsa
S-tyX

••
•
•a

W
5 5- aa- HaeeaX開発した手術戦略デスク• 機器の使用状況を「時刻同期」して表示・保存• メーカーの違う装置からのデータを「融合」して表示する

• SCOPE•• IOR SCOT
• SDN
•••
•

•

• SDN• 5SDNnn

WAN

VPNQoSnAC

B

IOR

21

n5nnnJGN SINET5FATS

SINET 5 +SINET 5 +C
B

H28PCRISE(JGN)

VM
(OFC)

SINET5SINET5PC

OF VLAN+Sec-chan VLANOFSOF VLAN+Sec-chan VLAN

OFS

JGN
Router

OFS

()

JGN
RouterJGN
RouterOF VLAN+Sec-chan VLAN

JGN
RouterOFS

H28  
Control: 333,
Data: 444

Control: 111,
Data: 222
Control: 111,
OF VLAN
Data: 222

OF VLAN +Sec-chan VLAN

Control: 333,Data: 444  SINETControl: 1992,
Data: 2055

OF VLAN + JGNSec-chan VLAN
Control: 1992,
Data: 2055

+Sec-chan VLANSINET 

SINETSINETControl: 1992,
Data: 2055

JGN

JGN


Control: 1992,
Data: 2055

RISE switch

RISE switch

RISE server

RISE server  



Control: 111,
Data: 222
Control: 111,
Data: 222

Control: 333,
Data: 444

Control: 333,
Data: 444
  


SINET
Control: 1992,
Data: 2055RISE(Core)ODENOS

JGN

Control: 1992,
Data: 2055

SINET
SINET
Control: 1992,
RISE
Data: 2055
(Core)JGN

VMControl: 1992,Data: 2055

RISE switch

RISE server

RISE server

C-PlaneODENOSaTremaDriver

JGNRISE switch

AODENOSAODENOS
AOpenFloMController

SINET VMi

ODENOS(OFC)

SINET5SINET5OF VLAN+Sec-chan VLANOFSOF VLAN+Sec-chan VLAN

OFS

OFS()

OF VLAN+Sec-chan VLAN

OFS

PC
IOR

RISE(JGN)

JGN
RouterJGN
Routeretc.

JGN
Router

Wi-Fi AP(OFS)

JGN
RouterABC• SCOPE•• IOR SCOT
• SDN
•••••

Issues and solutions of awake
surgery in IOR
IssuesSolutionsIncrease of risk factors causedby parallel medical devices

Incident detection

Difficult to recognition ofsurgical process for staff

Visualization of surgicalprocess

The implicit knowledge of skillof an expert surgeon

Visualization of implicit
knowledge
Focus point

For the recognition of surgical process is necessary,and thus, surgical process analysis and modelingresearch are employedPrevious study recognition of surgicalprocess in neurosurgerynSurgical process analysis andmodeling
Surgical Phases Detection fromMicroscope Videos by
Combining SVM and HMM(F. Lalys, 2011 [1])n Intraoperative Analysis of Task
Progress and Performance ofGlioma Surgery usingNavigation
Information(Nakamura, 2010[2])n[1] Lalys F, Riffaud L, morandi X, Jannin P (2010) Surgical Phases Detection from Microscope Videos by Combining SVM andHMM. International MICCAI Workshop, 6533: 54-62.
[2] Nakamura R, Aizawa T, Muragaki Y, Maruyama T, Tanaka M, Iseki H (2010) Intraoperative analysis of task progress andperformance of glioma surgery using navigation information. International Congress on Computer Assisted Radiology andSurgery (CARS2010), 5(1): S405-S406.

Recognition of surgical processfor visualization of implicit knowledgenRecognizing and analyzing of surgical process aimto visualize implicit knowledgenAnalyzing of surgical process including implicit
knowledgenn

In particular, the awake surgery for glioma includes implicit
knowledge
Target case: awake surgery for glioma

Issues: Recognition of surgical process for surgicalprocess
analysisn

Need to extraction of information to identify the surgical process
kinds of instruments, position of instruments,
position of tumor and brain structurePurposen

Proposal for identification and modeling of surgicalprocesses using pre-operative and intra-operative
information in awake surgery for gliomanCreate surgical process modelingStep1. Definition of surgical processes and creation of modelingn

Identification of surgical process
Step2. Extraction of brain structure to position of surgicalinstruments
Step3. Extraction of surgical instruments type
Step4. Identification of surgical process using extracted
informationn

Experimentation and evaluation of the surgical process
identification

An outline of surgical process identification
method for awake surgery of glioma
Surgical processes modeling

Identification of surgical processes
Identification ofExtraction ofsurgical feature information surgical process

Step1

P1
P2
P3

P4P5

Surgicalinformation

P7P6P8P
9P process

Step2
Extraction ofMR Imagebrain structureusing theposition ofsurgicalinstruments
Position of surgicalinstruments

Microscope images

Step3
Extraction ofsurgical
instruments
type

Step4
Identification of surgicalprocessusing
extractedinformation

Step1. Definition of surgical processn

Manually extracted feature of surgical process using intraoperative informationnnkinds of surgical instruments
Position of surgical instruments and anatomical information

Target range of surgical process

start

1~2h

0.5h

5~9h

Craniotomy

MRI
scans

Surgical for glioma

0.5h 0.5hfin
MRI Suturescanskinds ofinstruments

position ofinstruments

videoStep1. Definition of surgical process
nDefinition of 9 surgical processes with expert clinicians using intraoperative information

instruments
bipolar forceps
bipolar probe
dura scissors
none

Position ofinstruments
brain surface
In the brain
In the tumor
Outside the brain

SulcusNear the sulci
Gyrus
none

Phase A kind of surgical processes
P1 Next process preparations
P2 Verification of tip position of theinstruments
P3 Cortical mapping
P4 Evaluating brain function on
cortical mapping
P5 Coagulation and approach to thetumor
P6 Meninges incision
P7 Resection of glioma
P8 White matter mapping
P9 Evaluating brain function on whitemapping

Step1. Definition of surgical process
Target range of surgical process

start
n

1~2h

0.5h

Craniotomy

MRI
scans

20~30mSurgical for glioma

0.5h 0.5hfin
MRI Suturescans

P5: Coagulation and approach to the tumor
Surgical
instruments
bipolar forceps
bipolar probe
dura scissors
noneSulcus
Near the sulci
Gyrus
none

Position ofsurgicalinstrumentsbrain surface
In the brain
In the tumor
Outside the brain

Step1. Create of surgical process
Target range of surgical process

start

1~2h

0.5h

5~9h

0.5h

Craniotomy

MRIscans

Surgical for glioma

0.5h

MRI Suture
scans
P1

P1

P2

P3 P4 P3

P8P9P2P3P4P5P7P6P8P9Modelingfin

Step2. Extraction of position ofsurgical instruments on MR image

Pre-operativeimage- Position of surgicalinstruments- Kind of sulci- Kind of surgical instruments

Deformation
using Brain
VISA[3],[4]Deformedsulci image

Segmentation

Intra-operative
imageSegmentation ofIntra-operative image

Positional relationshipsof surgical instruments
and sulci on MR image[3] Liu Y, et al.., (2014) An ITK implementation of a physics-based
non-rigid registration method for brain deformation in image-guided
neurosurgery. Frontiers in Neuroinformatics, 8(33): 1-10. D. Riviere,
J. F. Mangin, D. Papadopoulos-Orfanos, J. Ma.
[4] T. Nonguchi et.al.,(2015) A surgical navigation system guided byintraoperative cerebral sulci recognition in neurosurgery J CARS, vol.
10 (Suppl 1): S277-278, 2015.
.

An outline of surgical process identification
method for awake surgery of glioma
Surgical processes modeling

Identification of surgical processes
Identification ofExtraction ofsurgical feature information surgical process

Step1

P
1
P
2
P
3

P4P5

Surgicalinformation

P
7P
6P
8P
9P process

Step2
Extraction of
MR Imagebrain structureusing theposition ofsurgical
instruments
Position of surgicalinstruments

Microscope images

Step3
Extraction ofsurgical
instruments
type

Step4
Identification of surgicalprocessusing
extractedinformation

Step3. Extraction of surgical instruments type
teaching images (250*4)

Input Images

bipolar forceps

bipolar probe

dura scissors

nonebipolar forceps?
bipolar probe?
dura scissors?
none?
Convolutional Neural Network(CNN)+Support Vector Machine (SVM)

yes.
no.
no.
no

Step4. Identification of a surgical process andcreation of a surgical modelnIdentification of surgical process using HMM(Hidden Markov Model)
n Input data : step1~step3 data

Step1

P1

Step2

P1

P2

P2
P3

P
4- Sulcus- Position ofsurgical instruments

P5P7P6P8P
9

Hidden MarkovModel(HMM)

P3

P4P5Now process

Step3

Surgical process model- Position and type of
surgical instruments

P7P6P8P9

Experiment and Evaluation of theIdentification of surgical process
nInput data (1case 120minutes)
Intra- and Pre- operative MR Images(512×512×200 [voxels])n Microscope Images (1920×1080×3 [pixel])n Positional information of surgical instruments (22 fps)
nnMethods
Identification of surgical process, proposed
n Check the true surgical process by manual operationn We compared the identified surgical processes with thetrue surgical processesnExperimental EvaluationnResult: Rate of surgical process identification(120 minutes) 77%
Identification of surgical process using bipolar prove: 94%(P3,P4,P8,P9)n Identification of surgical process using bipolar forceps:63%(P5,P7)
nP9 Evaluating brain function on whitemapping
P8 White matter mapping
P7 Resection of glioma
P6 Meninges incision
P5 Coagulation and approach to thetumor
P4 Evaluating brain function on corticalmapping
P3 Cortical mapping
P2 Verification of tip position of theinstruments
P1 Next process preparations

• SCOPE•• IOR SCOT• SDN•••••nnnn

MRIMRIMRInnn

MRIMRIMRI

1.

MRI
2.
3.
4.

MRI

1.
nnn

9mmnn

MRI2.
MRInn[*][**]nnnnn

MRIMRIn[*] Entropy minimization for automatic correction of intensity nonuniformity, J.-F. Mangin, MMBIA (Math. Methods in Biomed.
Image Analysis), Hilton Head Island, South Carolina, IEEE Press
162-169, 2000[**] A. Tristan-Vega, V. Garcia Perez, S. Aja-Fenandez, and C.-F.
Westin, "Efficient and Robust Nonlocal Means Denoising of MR
Data Based on Salient Features Matching", Computer Methods andPrograms in Biomedicine. (Accepted for publication) 2011.

3.
nnLiu

[*]

nMRIMRIMRI[*] Y. Liu, et al., “An ITK implementation of a physics-based non-rigid registration method for brain deformation in image-guidedneurosurgery”, Frontiers in neuroinformatics, vol.8, no.33, pp.1-10, 2014.

4.
MRnn

Statistical Parametric Mapping (SPM)[*]
nn

MRI
MRI

SPM

MRI
[*] SPM: Statistical Parametric Mapping, http://www.fil.ion.ucl.ac.uk/spm/. (last accessed at 2016/08/30)

SA : Speech Arrest•SI : Speech Impairment•B : Broca's area
•W : Wernicke's area•n, v, r :• n : naming• v : verb generation• r : reading
11nMRInnn

(61.

MRI

2.

MRIMRI(4~5 )

nnn:)

1.

MRI

MRI
3.1±1.5 [mm]n3.0±1.4 [mm]3.3±2.0 [mm]

nn

2.

MR
5.2±3.3 [mm]nnn

5.2±3.5 [mm]5.3±2.8 [mm]• SCOPE•• IOR SCOT
• SDN
••••

•

nnnnnSCOTnnnnnDB• SCOPE•• IOR SCOT
• SDN•••••nnn

Medicine 4.0

SCOTnnSCOTnnSDN

NWnnnnSCOT
Industry 4.0, Society 5.0

NW