From: sinster@cse.ucsc.edu (Darren Senn)
Newsgroups: sci.nanotech
Subject: Re: Multiple Experiences
Date: 20 Aug 92 00:18:59 GMT
Organization: University of California, Santa Cruz (CE/CIS Boards)

In article <Aug.15.10.19.21.1992.10779@planchet.rutgers.edu> hsr4@vax.ox.ac.uk (Auld Sprurklie) writes:
>From what I'd read so far, it appeared to me that the memories -are-
>associated with the means used to store them (don't different transfer
>functions apply to storage of different types of memory, some transfer
>functions being unsuitable for learning in certain cases ?), [...]

The transfer function is generally fixed when the network is first
created: it's not learned.  So the transfer function is really part of
the net's structure, not the memories.  Certainly, the exact values of
the weights and thresholds depend on the network's transfer functions,
but since only the weights and thresholds change, it's somewhat
meaningless to associate the memories (a purely learned phenomenon)
with the transfer functions (a static phenomenon).

> [...] and if
>the distribution -is- complete, as I had understood, that brings me
>back to my earlier query about the extraction (i.e. copying) of a
>skill or set of skills, presumably associated with a set of memories.

Well, each skill would be composed of a sequence of memories and simpler
skills (which are themselves composed of memories and simpler skills,
etc.).  At the core, a memory is an atomic skill, so at least this
recursion terminates  :).  Imagine the skill of writing the letter "a".
Presumably you start with a blank piece of paper, a pencil, and the
desire to do so (rather important, that last :).  The first association
in the skill will dictate a starting position for the letter.  Another
skill will get triggered to put the tip of the pencil there.  The next
association will dictate the first stroke, and that will trigger another
skill.  I hope you can see where this is leading.

So in order to copy a whole skill, you must discover all the memories
that compose that skill and its subskills, and teach those memories to
the recipient network.  Once you have the memory, teaching it is simple.
Finding the memories that compose the skill is a different matter
entirely.  :(

[...]
>[...]  The obvious
>question which follows on is: how do you decide which dimple or group
>of dimples corresponds to a particular skill/memory ?  And if we still
>take the parallel dump from clones to primary as a possibility, would
>it be possible to differentiate the new skills/memories from the set
>supplied to form the clone(s) in the first place ?

Detecting which dimple/group corresponds to a particular memory is simple.
Assert the memory onto the network (that means provide the network's
input _and_ output) and calculate the optimal set of weights for that
memory (there's a generalized formula for this).  Euclidean distance
between the memory's ideal output and the real output of the net is
the energy of that memory: the lower the energy, the better the
memory has been learned.  This tells you the energy at that point on
the energy curve.  If you want to find out the shape of the curve at
that point, then you'll have to examine the energy at neighboring
points by tweaking the real weights a bit.

The danger, when teaching a new memory to a network, is that old memories
might be forgotten, or just become harder to reach.  Imaging a stretched
sheet of silk with a number of marbles glued to certain positions.  These
marbles will deform the silk.  If you drop a glass bead, somewhere, it'll
roll around until it runs into a marble.  This marble is the memory
associated with the input corresponding to the bead's original placement.
If you put a bowling ball on this sheet of silk, then a large number of
marbles will never be hit by the bead.  The bowling ball represents a
memory that is remarkably easy for a particular network to learn, or
a memory that has a large input space (imagine teaching a network to
recognize individual faces -- the marbles -- and then training that same
net to recognize Caucasians -- the bowling ball).
-- 
Darren Senn                                            Phone:    (408) 479-1521
sinster@scintilla.capitola.ca.us                       Snail: 1785 Halterman #1
Wasurenaide -- doko e itte mo soko ni anata wa iru yo.     Santa Cruz, Ca 95062