Newsgroups: bit.listserv.csg-l
Date:         Thu, 20 Aug 1992 20:20:35 EDT
From:         mmt@BEN.DCIEM.DND.CA
Subject: Re: Little Baby & Reorganization

[Martin Taylor 920820 19:15]
(Bill Powers 920818.2300)

Before your long tutorial on reorganization got here, Jeff, Allen, and I had a
discussion on Jeff's proposal.  I hadn't fully understood it from the written
description, and I don't think you did, either.  But your tutorial helped us
to refine our understanding of your approach to reorganization.  Let me see
if I can rephrase Jeff's as I understand it. (If Jeff doesn't accept this as
his, then consider it a derived version that I think should be considered).

I'm not saying I buy Jeff's system, but I don't find it incredible, either.
It sounds workable, and has a good evolutionary rationale based on real
long-term consistencies in the environment.

Jeff starts with the proposition that I use in my version of reorganization,
that persistent and growing error in any ECS will cause THAT ECS to reorganize
its local environment.  There are all sorts of ways it could reorganize, none
of which imply any knowledge about what it is doing within the hierarchy or in
the outer world.  We all agree on the need for that kind of ignorance (although
I have, as we say, a "flag up" on that, for later discussion in connection
with symbolic AI).

Jeff incorporates a separate control system, which could be a small hierarchy,
that you call a "parasite."  I'd prefer to call it a "symbiote."  It is a very
primitive thing, in evolutionary terms.  It takes advantage of regularities
that do occur in the environment, such as that too much thermal energy disrupts
biologically active molecules, and that therefore it is a good idea for ANY
organism to move away from sources of high-level thermal energy.  The symbiote
doesn't know how to make this move happen, so what it does is to execute
(possibly random) actions in the motion effectors.  It knows where they are,
relative to the source of the unwanted energy, because it has evolved as part
of the organism's ancestors for long enough that sucessful symbiotes have
selectively survived.  But it doesn't know any more than to create some strong
action in,say, an arm or a leg.  It is not controlling in the sense we usually
use, with precision, but it is controlling in the sense that flailing around
quite often moves one away from a localized source of trouble, and that brings
the symbiote's percept into a range near its reference level. (Aside: control
to be away from some central point is a quite different problem from control
to be near the central point.  It's not just a question of reversing the
sign of the output.  It's the difference between "No" and "Yes" in language.
"No" means "Anything except what was proposed," a wide range of possibilities.)

What happens when the symbiote induces this relatively violent activity?
ECSs experience large and momentarily growing error if they have as part of
their feedback loop the actuators affected by the symbiote.  This (in my
view) is a condition that lasts for a moment, in which any affected ECS
might experience a reorganization event.  That might involve a change in
their perceptual input function or in their output connections, or it might
only affect their gains, or conceivably their reference input connections.
But it would be a local change.  These ECSs know nothing of the symbiote.
But so long as occasions arise in which the symbiote is activated, they will
reorganize, and thus will (if their perceptual input connections have the
information) become less likely to act in such a way as to activate the
symbiote.

The symbiotes are, as you point out, already set up to respond to particular
environmental states, or, more properly, to internal states that in many
cases have simple environmental correlates.  All the symbiotes can do is to
cause muscular activity in the world, or to cause the organism to emit
chemicals or to change its appearance (the scent of fear, facial flushing
or the colour change of a chameleon, for example).  They do not, as you
suggest, "teach another system to do it the same way."  They have primitive
stereotyped behaviour that is often immediately effective.  But the evolved
main hierarchy can learn much better ways of not getting into the bad
situation, or of getting out of it once there.

Gordon Liddy can burn his hand, because he has a high-gain (high insistence)
ECS for something of which that act forms a part.  High-gain systems can
over-ride the primitive symbiotes, but it is hard.  Most people couldn't
do it, except under extreme duress, where there are large deviations from
many conflicting reference levels (excuse the shorthand).

You are quite right that an acting symbiote reduces the degrees of freedom
available for the main hierarchy to act, possibly drastically.  But that
loss seems validated by experience.  We do not maintain our flexibility of
action in a "primitive stress" situation.  We tend to focus on removing the
stressful condition, which (with luck) the main hierarchy has learned to do.
Adults don't kick and scream when hunger gets extreme.  They try to find
food, with some determination and ignoring other tempting distractions, at
least until their systems lack the energy sources to keep going.  Babies
kick and scream, which, if Jeff is right, is the effect of the symbiotes.
Adults focus on control that works in a much more complex environment than
just the proximity of Mother.

The symbiote's job, as I see it, is not to signal "Pain" but to create error
in the hierarchy, especially in a part whose reorganization is likely to
result in useful control of whatever the symbiote does not like. "Pain" is
an interpretation of stimulus conditions accessible to the main hierarchy,
stimulus conditions that have been associated with action of the symbiote
in earlier days (and perhaps still).  There are many kinds of pain, and if
Jeff's notion is right, maybe they are associated with different symbiotes.
========================

I think we have three candidate reorganization systems that are plausible, and
I am happy to hear that this is what you are working on right now.  We may
actually be able to help with real (simulation) tests before too long (not
me personally, but my faithful contractors).

The three, in cartoon form:

1) Powers: Intrinsic variables (such as body chemical states) have reference
levels fixed by evolution.  Errors in them induce a tendency to reorganize
in the CNS hierarchy.  The actions of the CNS hierarchy affect the levels of
the intrinsic variables only as side-effects of whatever happens in the outer
environment.  There may possibly be some localization of the reorganization
to the upper levels of a growing hierarchy, leaving lower levels relatively
immune.  As the hierarchy grows, new, labile ECSs are added on top of
existing ones, but layers can grow laterally as well.

2) Taylor: Intrinsic variables participate in a single hierarchy along with
the CNS.  The ECSs that have intrinsic variable levels as (part of) their
perceptual input functions are always at the top level of a building
hierarchy.  Persistent and growing error in any ECS induces reorganization,
which may include the generation of a new ECS that receives its reference
signals from the reorganizing one and that gets its perceptual input and
sends its reference output to places connected to the reorganizing ECS.
Other aspects of reorganization include the making, breaking, and reweighting
of output-to-lower-reference connections, and Hebbian modification of
perceptual inputs.

3) Hunter: Intrinsic variables provide inputs to primitive symbiote "control"
systems that have the main function of inducing error in ECSs in the main
hierarchy when the intrinsic variables depart far from their reference
levels.  Reorganization in the main hierarchy proceeds roughly according to
the Taylor model, except that the top-level ECSs are not connected with
intrinsic variables, and new ECSs are "grown" on top of the hierarchy, in
the same way as the Powers proposal.
========================

> I'm plugging away with tests of various kinds of
>reorganizing in simple control systems just to find out what happens when
>you do it in different ways. Some ways are terribly slow; some work so fast
>that I'm suspicious of them. I've already ruled out one: reorganizing by
>switching the polarity of the output effects from 1 to minus 1 doesn't
>work, because it causes tremendous transients that can flip the system into
>unrecoverable states. Maybe if I stick a 0 in the middle as a third
>possibility it will work better. Or maybe output reorganization has to
>occur on a continuum between 1 and -1. We have to find these things out
>before we can come up with a workable system

The Little Baby is intended to provide a reorganization testbed, with a view
to using its findings for developing much more complex networks for dynamic
perceiving systems--speech recognition is the target.  Chris Love is building
it, not Jeff and Allan, but they do talk together.  I wish you had a Macintosh,
for 2 reasons: it is so much easier than a PC to work with, and we could
communicate programs.  We are using the Object-Oriented visual dataflow
language Prograph.  It's a bit hard to convert between that and a text-oriented
language like C, though algorithmic concepts and formulae are relatively easy.

We would love to know anything you find out about reorganization from your
experiments.

Maybe the transient when an output changes from -1 to 1 could be avoided if
the gain dropped during the reorganization time for a period close to the
inverse bandwidth of the feedback loop?  Or perhaps the problem is that your
hierarchy is too symbolic, in the sense that a given ECS has a function very
distinct from all the others at its level, and therefore has only a very few
connections at a lower level?  If an ECS provided references to many lower
ECSs that had highly correlated perceptual input functions, then the flipping
of the sign of one link would not produce a large transient.  It would
be much like having a continuum of output weights, and I think it might also
provide for more flexibility in the final (reorganized) system.  But it would
be computationally expensive to simulate.

Just a couple of suggestions.


Martin
Comments: Gated by NETNEWS@AUVM.AMERICAN.EDU