From: erwin@trwacs.fp.trw.com (Harry Erwin)
Newsgroups: comp.ai.philosophy,sci.cognitive
Subject: Wetware
Date: 22 Jul 92 12:40:11 GMT
Followup-To: sci.cognitive
Organization: TRW Systems Division, Fairfax VA


First, I'd like to thank everyone who responded to my recent postings. The
question about David Marr's theory of vision got the following responses:

(Marc Green) "Zero crossings have long been given up as ambiguous and
inaccurate." "The ... worthwhile notion in his theory is the use of
multiple scale operators. And that had already been a long standing idea
in visual psychophysics and physiology."

(Hans du Buf, responding to my question on whether Marr's proposal to base
a theory of vision on the identification of zero-crossings has held up)
"Yes and no. The main problem with zero-crossing scale space is that the
zero crossings are caused by >something<. One can track zero-crossings
from a coarse scale to a fine scale (Bergholm, IEEE PAMI, "Edge
focussing") or look for scale intervals where zero-crossings remain more
or less stable (Caelli and Bischof, "stability analysis"), but that
>something< is the problem."

"More recent detection methods are based on complex Gabor filters
(cortical simple cells) or other quadrature filter pairs, detecting local
extrema in the modulus of the response or equivalently the maximum phase
congruency over filters tuned to different frequencies. Such a method can
detect positive/negative lines/edges and trapezoidal edges, but
1. these detection strategies are impaired by interference,
2. require a higher multi-scale analysis, and
3. one is unable to detect the extrema or zero-crossings of a sinewave
grating.

"This is not an easy subject, and I have the impression that we are now
developing >real< models of low-level vision, more than 10 years after
Marr's death. See, for example, Heitger et al., in a recent Vision
Research.... There are many approaches studied in the literature, e.g.,
optical flow and disparity based on local phase information. But a
complete processing scheme is still lacking, and the simplest problems are
still tough."

On my comment on the Malcolm P. Young paper, Steve Barry expressed a
concern that Young was going too far beyond the evidence he was citing.

I got some interesting responses to my note, "The Brain as a Submarine
Combat System." Bill Skaggs pointed me at Barto's paper "Connectionist
Learning for Control: An Overview," in Neural Networks for Control,
Miller, Sutton, and Werbos (eds.), The MIT Press, 1990. That paper is more
oriented towards applied neural networks than towards wetware, but it does
identify some mechanisms for getting around the "infinite regress" problem
in training. Barto points towards Werbos' Heuristic Dynamic Programming as
one interesting approach to reinforcement learning. Paul spoke on that (or
equivalently adaptive critic systems) in his luncheon speech at the
ANNA-91 conference, and I've heard him discuss it more informally in the
Georgetown colloquia, but I had never realized that he was specifically
proposing that the brain may implement a dynamic programming algorithm to
get around the planning problem.

I know a little about dynamic programming, having reinvented it
independently about 1982 to help me analyze the non-zero-sum game with
information collection in a biological setting. Dynamic programming was
defined by Richard Bellman in the 1950s as an approach to converting a
problem involving optimization at some delayed time horizon or over some
interval into a problem of optimization at each intermediate time point.
For a fixed time horizon, the technique involves solving backwards from
the horizon to the current time. If the time horizon is at infinity, the
problem is more intractable, and you have to either use renormalization
techniques or use approximate solutions for a finite arbitrary horizon and
let the horizon go to infinity. Dynamic programming has been described by
a expert as "jillions of special cases."

In the work I did, I demonstrated that the one-sided game against nature
did have a nice solution technique, and so I can believe that Werbos'
approach is probably implemented biologically. It provides an effective
way of calibrating a preexisting stimulus-response system to actual
conditions. However I also demonstrated that DP approaches failed for the
2+ person game and that the population strategy could be expected to
evolve chaotically. I'll get back to the implications of this in a bit.

Werbos' method (Werbos, 1990, "Consistency of HDP," Neural Networks, V. 3
#2, 179-189.) involves two or three parallel networks. One of the networks
evaluates the current state of the system and calculates a representation
of the strategic optimization function (the function that evaluates the
value of each current action relative to the strategic goal). The second
network is the action network, which selects the action to be performed
based on the current state and the current strategic optimization
function. The third (optional) network is a dynamic model of the
environment. Werbos identifies a way of training the entire system. Note
that this approach defines a way to model the preplanning seen
particularly in mammals. 

It appears to me that HDP may describe how learning occurs in lower
mammals. One concern I have, though, is that I doubt it is effective in
training social behavior. This concern emerges from the difficulty I had
using DP approaches to solve social games. I suspect multiple parallel HDP
networks are needed at the very least.

Neil Rickert also had some interesting comments. He points out that it
appears that the human brain keeps track of and analyzes (in some sense)
all sensory input, not just "targets of interest." This results in a
system that can recognize some input as a pattern, purely on the basis of
its similarity to input that has been frequently recurrent in previous
experience. He suggests that we should consider how to build a pattern
detection/ pattern recognition system, since it seems basic to minds. He
also suggests we look at how the mind directs its attention and sets its
goals.

Christopher Ian Connolly suggests that I should look at the way the basal
ganglia interact with my "motor beams" since they may be the actual
generators of those beams.

I also promised to post my latest ideas on this "beam" model of
consciousness.

1. There is an autonomous cognitive beam in the human brain. Minsky points
out that self-awareness continues even when sensory input is cut off. This
autonomous cognitive beam is almost certainly chaotic in its dynamics. It
is also robust, being resistant to linear and non-linear perturbation in
normal individuals, suggesting that the chaos is structurally stable (with
all that implies).

2. I'm currently looking into the following issues:
 a) How automatic abstraction can take place.
 b) The nature of the self simulation in the temporal cortex (HDP?).
 c) The role of internal drives in generating various beams (basal
ganglia?).
 d) The brain structures needed to support social behavior.

3. I'm also trying to define the various possible ways beams can be
created and trained.

Cheers,
-- 
Harry Erwin
Internet: erwin@trwacs.fp.trw.com

Newsgroups: comp.ai.philosophy