Newsgroups: bit.listserv.csg-l
Date:         Fri, 9 Oct 1992 17:09:51 EDT
From:         mmt@BEN.DCIEM.DND.CA
Subject: Re: reorganization

[Martin Taylor 921009 17:00]
(Bill Powers 921006.1530)

> Just by trial and error, I found that the most
>reliable method (so far) entails (roughly) making the rate of
>reorganization depend on the rate of change of absolute critical error
>TIMES the absolute critical error, so that the changes get smaller as
>the remaining error diminishes. I realized only afterward that this
>amounts to taking the derivative of the SQUARE of the error, which
>neatly takes care of assuring positive values and that the result will
>give the best least-squares fit to the optimum solution. You might try
>this in addition to your (e*e + k*e*de)/G.
>In other words, try d(e^2)/G.

Did you try varying the relative contributions of the value of e**2 and
of d(e**2)/dt?  In other words, varying k in my expression between zero
and infinity?  My formula is yours if k is large, but if k is moderate, then
reorganization could occur with a sustained error that does not change
value, which seems as if it would be a good thing to do.

>Another tip. Rather than just make random corrections based on
>critical error, I have found that it's best to randomly change a
>parameter that determines the direction and speed of changes in
>parameters. Between reorganizations, the changes then continue on each
>iteration. This is strictly analogous to the E. coli method of
>locomotion, in which, between tumbles, the organism continues to swim
>in a straight line, continuously altering its relationship to a radial
>gradient.

This sounds as though you are doing Hebbian learning (on the output link
weights?) rather than what I characterize as "reorganization" (the changing
of link sign or of the link targets).  I can't see what this paragraph
would mean when applied to output links that had weights of only +-1 or zero.
I thought that the simple control nets you were working with all had this
restriction on the output weights, and that you were waiting to be forced
into allowing real-valued output weights.

My intuition has been that you would be forced into using real-valued
output weights when you got into massively parallel control nets in which
no single ECS could be responsible for the whole of any behaviour and there
was permanent tension in the net.  But so far as I am aware, you haven't
studied that kind of net yet.  Am I wrong? Or did you get into real-valued
weights when you found that sign-flipping had nasty effects?  Or are you
talking about perceptual input functions here?

>>In the Little Baby project, the second (probably) experiment will >be
>>to start with a random set of output connections, and use Hebbian
>>learning on the perceptual input functions to see whether the baby
>>can learn to perceive the world in a way compatible with its (fixed
>>randomly assigned) outputs.
>
>I can tell you already that this will work, using my method of
>reorganizing outlined above, with up to 20 independent control systems
>controlling through 20 shared environmental variables. The critical
>variable is total squared error across all systems. Reorganization is
>applied globally, not based on each system's error.

Again, I'm not 100% sure what you were varying.  I had assumed that your
reorganization experiments were restricted to changing the output system
rather than the perceptual input system.  If you have studied error-induced
modifications of the perceptual input functions, we would love to know more
about your results.  We don't mind re-inventing wheels, but it is nice to
know that our wheels work like yours if we do.  (By "we" I mean Chris and me,
though I expect the CSG-L readership would be interested, too).

Martin
Comments: Gated by NETNEWS@AUVM.AMERICAN.EDU