Newsgroups: bit.listserv.csg-l
Date:         Mon, 5 Oct 1992 12:43:54 EDT
From:         mmt@BEN.DCIEM.DND.CA
Subject: Re: Learning and memory

[Martin Taylor 921005 11:30]
(Bill Powers 921005.0730)

This is in response to the discussion between Bill and Greg.

To start with, both have produced a set of principles, but neither gave a
functional description of an ECS and its place in the hierarchy.  To have
a clear discussion of learning, we have to have agreement on what the
hierarchy looks like, at minimum.  So here is what I have understood to
be more or less agreed.  If any of this is wrong, it affects the discussion
and should be put right.

As I understand the hierarchy, the unit of which all is built is the ECS.
The ECS has two kinds of input function:
(i1) a perceptual input function, which combines all the many sensory
inputs according to some algorithm and produces a scalar value called the
perceptual signal.  The word "sensory" includes the possibilities of direct
input from sensor systems and the perceptual signals produced by the perceptual
input functions of other (lower) ECSs.
(i2) a reference input function, which combines action output signals from
other (higher) ECSs.  Typically, this function is considered to be a simple
summation, but it could be any algorithm.  The output range of the reference
input function must not exceed that of the perceptual input function.  The
output of the reference input function is a scalar value called the reference
signal of the ECS.

The perceptual signal is compared with the reference signal to produce a
scalar value called the error signal.  The error signal is transformed by
an output function, typically but not necessarily an integrating amplifier,
to produce an output signal.

The ECS has two kinds of output that are distributed to other ECSs (except that
in the case of the lowest level ECSs the output goes directly to muscles of
other effector systems.  In each case there is the possibility that the output
is weighted differently in transmission to each individual destination:
(o1) A set of perceptual outputs that consist of the perceptual signal possibly
mulitplied by some weight.  These are the sensory inputs of other (higher) ECSs.
(o2) A set of action outputs that consist of the output signal possibly
multiplied by some weight.  These are the reference inputs of other (lower)
ECSs.


The ECS has the possibility of storing one or more values of its perceptual
signal and of using a stored value in place of its reference signal (episodic
memory).

The ECS has the possibility of using an action output signal as one of its
sensory input signals (imagination).

The hierarchy consists of ECSs linked only by the connection of the (possibly
weighted) perceptual signals of lower ECSs to sensory inputs of higher ECSs,
and of the (possibly weighted) action output signals of higher ECSs to the
reference inputs of lower ECSs.  Each kind of link is one-to-many.
===================

If the foregoing is a correct description of an ECS and its place in the
hierarchy, what opportunities are there for learning?  The following all
seem plausible:

Within a set of ECSs already in existence:

(1) Alteration of the perceptual function.
(2) Alteration of the perceptual-sensory link structure.
(3) Alteration of the output function.
(4) Alteration of the action-reference link structure.
(5) Alteration of the reference input function.
(6) Alteration of the content of the internal memory of the ECS.

And
(7) Incorporation of a new ECS into the hierarchy.

There are subclasses:
(1a) (3a) and (5a) Modification of the parameter values of the function
(1b) (3b) and (5b) Alteration of the form of the function.

(2a) and (4a) Alteration of the connection weights
(2b) and (4b) Alteration of which ECSs are linked.

All in all, this makes 12 logical possibilities for types of ways the hierarchy
can learn.  Not all are effective, and the discussion has generally focussed
on only a couple.

Hebbian learning is a term usually denoting topologically smooth changes
of parameter values in a combining function.  There are many forms, but
generally speaking they all involve changes based on some measure of goodness
of the present set of parameter values in relation to the data input to the
function.  One way is to make more extreme a pattern that creates a large
output, and to make less extreme a pattern that produces a small output.
Another way is to move the input parameters in such a way as to increase
the output when some "teacher" asserts that the input pattern is one to
which the function "should" give a large response.

Within the ECS hierarchy, Hebbian learning has usually been discussed as it
applies to the perceptual input functions (1a), and then usually when the input
functions are seen as nonlinear compressions of weighted sums of their
inputs.  If the hierarchy consisted only of ECSs having this form of perceptual
input function, the perceptual side would be a classical multilayer perceptron,
and Hebbian learning would suffice to allow it to perceive (and hence possibly
to control) any describable partitioning of the sensory input data space.
Other forms of perceptual input function, possibly involving delayed sensory
inputs, are possible and may be necessary for the control of dynamical
percepts such as sequences.

Hebbian learning could apply in classes 1a, 3a, or 5a.  Because the success
of control does not depend much on loop gain if the loop gain is high enough,
one would not expect it to be very important in 3a, except in adjusting
parameters such as integration time constants.  It might be important in 5a,
which affects the relative strength of different higher-level ECSs on the
reference signal of the ECS in question.  No discussion (that I remember) has
considered this possibility, or how it might work if it happened at all.

Hebbian learning could apply in 2a and 4a as well, but numerically the
results would be indistinguishable from 1a and 5a applied in a different ECS.
The reason these possibilities are listed is that there is a potential
question about the location of responsibility for the alterations of weights.
"What does the ECS know, and when does it know it?"

Hebbian learning cannot be relevant to structural alterations in the hierarchy
(1b-5b), because the learning is not topologically smooth.  This is the
province of "reorganization."  It also cannot apply to classes 6 or 7, which
involve discrete events that change either the content or the structure of
the hierarchy.

(Powers)
>Hebbian learning makes parameters depend on long-term effects of the
>signals passing through the network that is being reorganized; that's
>the Hebbian version of reorganization. For this to work in general, it
>must be true that there is some "best" way of handling signals. This
>assumption shows up in the postulate that the output of a neuron
>somehow "strengthens" the effects of input signals that exist at the
>same time as the output signal. The implication is that it is best for
>the organism that all signals contributing to a given neural output
>have the maximum possible effect. This sort of rule, I believe, is an
>attempt (of which I approve) to get away from a "teacher" that already
>knows how a neural function should be organized.

This is indeed one form of Hebbian learning, but the motivation is not to
get away from a teacher, so much as to provide the maximum discrimination among
input signals that is consistent with the variation in stimulus patterns.
Usually, weights on low-valued inputs decrease when weights on high-valued
ones increase, so as to sharpen the discrimination.

> Any system in which there is a preferred kind of
>input or output function loses the ability to adapt to environments in
>which some other kind of input or output function is required for
>successful control.

True.  And I think that your "levels" of ECS accommodates that.  But Hebbian
learning can operate within different kinds of input function, provided that
the function is such that small changes in parameter values cause small
changes in the function's behaviour.

I had intended to carry on this posting with a discussion of reorganization
possibilities, but I think it is long enough, and I have spent too long on
it, already.  Maybe later.  But if you accept my categorization of learning
possibilities, there seem to be 5 kinds of reorganization, of which perhaps
1 or 2 may be useful in learning.  And then there are episodic memory learning
and the introduction of new ECSs.  So there are many possibilities for ways
in which the hierarchy might change to develop new skills.

When there are so many possibilities, it makes good sense to see what happens
when only the most prominent are used.  In this case, I think that the most
likely candidates for effective learning are (in no particular order),
1a, 2+4b (together), 6, and 7.  These can be verbalized as "What can I perceive"
"What can I do to control what I perceive" "What have I perceived" and "I
can do nothing right--let's try something completely different."

Martin
Comments: Gated by NETNEWS@AUVM.AMERICAN.EDU