コネクトームと人工知能
Connectome & AI

水谷治央 (Haruo MIZUTANI)
Molecular & Cellular Biology
Harvard University

Connectomics• Connectomics aims at deciphering all (synaptic) connections
between all neurons and clarifying wiring diagram of theneuronal networks.
• Scale of Connectomics– Microscopic• Electron Microscopy－Macroscopic
・ MRI (Diffusion Tensor Imaging)・ Light Microscopy

The brain isorganized oversizes that span 6orders ofmagnitude
An MRI voxel(1mm3) is 1 trillion
times larger thanan electron
microscopy voxel(100nm3)

Lichtman & Denk 2011

Scale: Devices & Resolution

Workflow for large-scale neuron reconstruction

Plaza et al. (2014). Curr Op Neurobiol 25, 201-210.

3D EM techniques

Retina neural circuit reconstruction
SBF SEM

Helmstaedter et al., Nature 500, 168-174 (2013)

Automatic Tape-collection Mechanism
for Ultramicrotomes• Lossless & relatively damage freesectioning• Nondestructive sectioning• Stable and sturdy after sectioning• allows for:• Immunohistochemistry (theproteome meets the connectome)• Multiscale imaging (same section
re-imaged many times at differentmagnifications)• Post sectioning intensification
methods for added contrast• Multi-instrument image acquisition
multiplexing or more exotic
solutions for speed

Ken Hayworth and Richard Schalek

ATUM cutting & collecting sections

ATUM(AutomaticTape-collecting UltraMicrotome )Commercially
Available in
Boeckeler
Instruments, Inc.

Hayworth et al., 2014

WaferMapper

Code for freehttps://wafermapper.googlecode.com

Hayworth et al., 2014

30 nm/pixel6 nm/pixelComputerassisted “VAST”
saturated handsegmentation ofcortex 6 x 6 x 6μm
216 μm3 =2 months

Daniel Berger.

We began 7+ years ago:
0.5 M pixels/sec=
2.24 centuries for 1 mm3 (~2PB)
So far:40 M pixels/sec =2.8 years for 1 mm3Now:
1-2 G pixels/sec =3~6 weeks for 1 mm3

61 multi-beam
SEMSingle Hexagon Image
1.3 S

3.8nm pixel size
100 ns dwell time
430 pA per beam
470 Mpixel per second
10μmLarge-Area Imaging
Fully automatic run
492 individual hexagons
2 x 1.6 mm sample size
4 nm pixel size
290 GB file size
100 ns dwell time
~60 min imaging time

1 mmUltrathick sectioning (20 μm)

30 nm
lostHayworth et al.,Nat Methods,12:319-22 (2015)

Whole mouse brain EM imaging

Mikula & Denk @ Max Plank
Nat Methods.12:541-6 (2015)

80nm/pixel resolution

Whole mouse brain• Mouse-brain volume: 0.5ml• Number of voxels at 80 nm isotropic resolution: 1015
• (500 hard disks, $100 each -> $50k)• Imaging time at 40 MHz: 280 days

• Number of voxels at 20 nm isotropic resolution: 6x1016• (30,000 hard disks, $100 each -> $3M)
• Imaging time at 40 MHz: 50 years• 5nm isotropic resolution: 3.6x1018• (1,800,000 hard disks, $100 each -> $180M)• Imaging time at 40 MHz: 3000 years

Whole mouse brain• Mouse-brain volume: 0.5ml• Number of voxels at 80 nm isotropic resolution: 1015
• (500 hard disks, $100 each -> $50k)• Imaging time at 2 GHz: 6~10 days

• Number of voxels at 20 nm isotropic resolution: 6x1016• (30,000 hard disks, $100 each -> $3M)
• Imaging time at 2 GHz: 1~2 years• 5nm isotropic resolution: 3.6x1018• (1,800,000 hard disks, $100 each -> $180M)• Imaging time at 2GHz: 60~100 years

Challenges for Data Acquisition• Speed– 1-2G pixel/sec, 6 weeks for 1mm3 @4x4x30nm– Whole mouse brain: 1 year with 400 beams, 7 mSEM

• Resolution– 6.25 x speed at 10nm/pixel, 1 week for 1mm3
– Whole mouse brain: 2 months with 7 mSEM

• Cutting with big diamond knifes– Hot diamond knife– Larger than 1 cm diamond knife– Replacing knife• Deep stainingFully automated segmentation

Deep Learning for the Connectome
(a)

(b)

(c)
4-layer deep network(a) Original EM image(b) Deep network output(c) Segmentation pipelineoutput
http://www.rhoana.org/

Full Automatic vs Hand Segmentation
Peril: still not quite good enough without substantial human editing

Fully automatic (FUSION)

Hand segmentation (VAST)
Kasthuri et al., Cell 2015
Seymour Knowles-Barley with Hanspeter Pfister et alMojo: Connectome Annotation Tool

http://www.rhoana.org/

Challenges for Data Analysis• Very computationally intensive– 38 million flops per output required to process
image data at 0.85TB/day• 250 nVidia Tesla K40 GPU ($2M: $8,000/K40)• Learning algorithm errors– 0.9/um3 split human operation required to correctfor merger errors– 5.8/um3 merge human operation required tocorrect for split error• Proof Reading– Intensive proof time

Challenge on 3D segmentation

http://brainiac2.mit.edu/SNEMI3D/

EyeWire: neural annotation gameEyeWire DomoGlobal gamer density• Japan: 63/100 gamers• USA: 55/100 gamers• Europe: 13/100 gamers

0.14 yeas per day

Scaling up• terabyte• petabyte• exabyte• zettabyte(0.1 mm)3 retinal column
1mm3
cortical column
1cm3mouse brain
1000cm3 human brain• Angry Bird
– 600 years per day (5 million hours per day)• EyeWire– Only 2 days required for 1mm3 annotation• If gamers would play as much as Angry Bird

Saturated reconstruction (>1500 cubic microns) of cylinders surrounding two nearb
apical dendrites

Three-cylinder decomposition by type

Daniel Berger

What’s in the cylinders?
• 1407 axons– 94% excitatory– 5% inhibitory– 1% ?

• 193 dendrites (~7-fold fewer than axons)– 91% excitatory “spiny”– 9% inhibitory “smooth”• 1700 synapses in 1500 mm3 (1.13 synapses/mm3)– 92% excitatory• 92% on spiny dendrites• 8% on smooth dendrites– 8% inhibitory• 93% on spiny dendrites• 7% on smooth dendrites

N Kasthuri, D Berger, et al

Database of all the axons
in red cylinderSynapse database (1700 synapses)Connectivity Clustering: Multiple spines of
one dendrite innervated by the same axon

Connectivity Clustering: all theB Kasthuri
D. Berger
D Lee

Real Brains in the Computer
Synapse Catalogs
AnatomyForm of neuron
Size of neuron
Number of neuron
Length of axons & dendrites
Projection of axons

Function
Electrical properties
Spike timing
Transduction speed
Transmission efficacy
Neurotransmitters
Receptors
Expressed genes
Physiological significanceDisease (dysfunction)

コネクトームの応用先（?）• 特定の技術の運用が上手い人と下手な人– 自動車の運転– 作曲、絵画– スポーツ選手• 病理が難解な神経疾患の機構解明– 統合失調症、うつ病、自閉症• アインシュタインの脳解析– 天才性の追究Jeff Lichtman Lab•••••

Bobby Kasthuri (Argonne/UChicago)
Daniel Berger
Richard Schalek
Seymour Knowles-Barley
Josh Morgan•
••
•
•
•

Ken Hayworth (Janelia)
Dong-il Lee
Allysa Wilson
JC Tapia (Columbia)
David Hildebrand (Engert)
Verena Kaynig (Pfister)

NIH
NIMH (Conte Center)
Dept Army (MURI)
HFSP
IARPA••••••••

Hanspeter Pfister
Nir Shavit (MIT)Art Wetzel (CMU)
Aravi Samuel
Florian Engert
Takao HenschDavid Clapham
Michael E Greenberg