From: dstampe@psych.toronto.edu (Dave Stampe)
Newsgroups: sci.virtual-worlds
Subject: Re: TECH: Neural Interfacing
Date: 11 Dec 92 15:34:28 GMT
Article-I.D.: u.1992Dec12.041658.16932
Organization: Department of Psychology, University of Toronto
Originator: hlab@stein.u.washington.edu


kahn@Csli.Stanford.EDU (Ken Kahn) writes:
>
>I recall seeing a film in the 70s (this was before video tape) of an SRI demo
>which could distinguish about a dozen mental states (that people could be
>train to put themselves into).  Someone with a helmet was thinking to control
>a cursor in a maze (up, down, left, right).  It was a neat demo.  Low
>bandwidth though.  About 2 or 3 bits per second. Maybe 15 years later Fujitsu
>will go somewhere with this.  I've been surprised that I haven't heard any
>followups, at least for the severely handicapped.
>
There seems to be a lot on misinformation about neural connectivity
going around, so let me add mine (B-{))

First, let's survey the techniques used so far: the up/down computers with 
pattern recognition and scalp electrodes, multi-electrode EEG, direct
cortical contacts, nerve interface chips, and high-resolution NMR.

The EEG and control computers use measurements from electrodes on the scalp,
thus average millions (nearly billions!) of neurons' outputs.  This is 
equivalent to debugging software by putting an AM radio next to your 
computer keyboard and listening to the noise: it gives you clues but 
nothing really detailed.  Even arrays of 40 or more electrodes do not
help too much: they can indicate some underlying mental activity that
involves large groups of neurons (alerting responses, preparation for
movement, and so on) but don't show anything really fine.  Basically, the
"control" systems that try to get simple commands from EEG rely on the
subject to do something unnatural internally to produce a large enough 
EEG response for the computer to pick up.  A simple one: blink your eyes.
You can measure a pulse of alpha waves (10 Hz) in the occipital scalp area.  
But then this is just 1 bit of information, and physically blinking your
eyes is much easier to detect.

High-resolution NMR (nuclear magnetic resonance imaging) can detect 
smaller areas of neural activity: nearly down to 1 millimeter square.
There are two problems with this (at least at present).  First, the 
brain state must be held for at least a second to be picked up.  Even
then, the noise level is so high that in brain research 20 or more
sessions must be averaged to get a "significant" result (1 bit error
in 20).  And NMR machines are huge-- they have to be to create the 
enormous and highly controlled magnetic fields they use.  I'm not
saying that future technology won't fix this, but nothing's on the 
25-year horizon.
The underlying problem is that neural activities in the brain seem to be 
divided at (at least for our purposes) two levels.  First, activity is
clustered into cortical "columns", about 1 millimeter square and 4 to 8
mm thick.  The activity in these can give simple clues (for example, area
of attention), but not much more.  Second, within each column (? The jury's
still out on whether this occurs between columns as well) data is 
represented by the activity of millions of individual neurons.  The form
of this data is diffuse and cannot be seen without direct cortical probes
or some other method-- even electrodes directly on the brain surface
will not be able to get all of it.

So how about direct brain contact (chips or electrode arrays)?  First,
it's still surface contact, so is not specific enough.  Second, penetrating the 
cortex with pointed electrodes is definitely antisocial, since nerves, 
neurons, and possibly blood vessels are being damaged.  Not something you
want to do more than (or even) once.  Even given nanotech (and so far most
of that is glorified science fiction speculation-- remember how SF picked
up on stuff like Tipler machines for FTL or time travel?) you still face
other problems.
The main problem with direct neural contact is interpertation.  On a
local level, neural patterns are nothing like a map of the world or
even a map of stimulus characteristics.  Usually you have a lot of 
"noise" in the position of elements in the map (after, the brain isn't
printed-- it "just grew").  Also, several types of information processing
are usually intermixed in stripes, columns and cortical layers.  So what
you need is a computer with 10^8 inputs (or more) to learn the current
connectivity of the brain area of interest, and translate data to/from
the brain representation.  I leave the training of this as an exercise to
future generations. (B-{))

Now for the real problem with brain connections, and the reason why I
feel that the current external stimulation method of VR will be the 
only real method.  Everyone seems to assume you can push signals 
to or from the cortex of the brain and everything will work.  Nothing
could be more wrong!  
Every sense (except smell) goes through extensive neural processing by or
is processed in parallel by important subcortical structures.  These also
happen to be the most inaccessible parts of the brain.  They are 
responsible for much of our "fluency" in viewing and interacting with
the world, and just inserting input at the cortex excludes them.  The
results from direct cortical stimulation will be unnatural and may
be closer to hallucinations than sensory input.  So you won't get real
VR this way-- another way to interface, perhaps, but not as useful. 
For example, let's say you're controlling a machine, and something
unexpected happens in some corner of the area.  Without subcortical 
control, your reaction time will be very slow, and you may not even 
notice it occuring!
As a real-life example, some researchers tried disconnecting part of the
somatosensory (skin sensation) cortex from input to control pain in 
terminally ill subjects.  Not only did this not work, but the pain became
harder to localize and thus harder to ignore!  Why?  Because other 
connections to the limbic system (a deeply buried section of cortex, 
highly involved with emotion and control) also gets connections to
pain input through subcortical structures.  (Damaging the limbic system
isn't an option, for obvious reasons).  The same type of warning goes for
vision, where subcortical structures are very important for the control of 
attention, noticing changes in the scene, spatial localization, and so
on.  Audition is even worse-- what gets to the cortex is so heavily 
preprocessed that cortical stimulation probably would be difficult (at 
least if you're looking to get 100% "vericidal" sound sensation.

So-- what IS possible?  I think that neural connections to the cortex are
interesting for research (in fact, essential to understand the brain).  But
for VR, you want to get realistic interactions and stimulation.  The
last place you can do this is where nerves enter the brain (even the
spinal column is a bit iffy, because we don't know all the processing 
that goes on in it).  Because nerves aren't well matched spatially, 
you'd need a computer to learn the connectivity and map your input
and output to the nerve.  How to connect in without damage?  Somebody
else's problem, I suspect.


--------------------------------------------------------------------------
| My life is Hardware,                    |         Dave Stampe          | 
| my destiny is Software,                 | dstampe@psych.toronto.edu    |
| my CPU is Wetware...                    | dstampe@sunee.uwaterloo.ca   | 
| Am I a techno-psychologist, or just an engineer dabbling in psychology?|
---------------------------------------------------------------------------
Newsgroups: sci.virtual-worlds