From: hsr4@vax.ox.ac.uk (Auld Sprurklie)
Newsgroups: sci.nanotech
Subject: Re: Multiple Experiences
Date: 15 Aug 92 14:19:22 GMT

In article <Aug.14.00.12.54.1992.412@planchet.rutgers.edu>, sinster@cse.ucsc.edu (Darren Senn) writes:
> In article <Aug.9.20.33.10.1992.15549@planchet.rutgers.edu> hsr4@vax.oxford.ac.uk (Auld Sprurklie) writes:
>>In article <Aug.6.20.51.51.1992.8765@planchet.rutgers.edu>, sinster@cse.ucsc.edu (Darren Senn) writes:
>>>[...]
[more ...]
> 
> The memories (associations) stored by a neural network are very strongly
> distinguished from the means by which those memories are stored (the
> connection weights, thresholds -- which are just connections to always-on
> neurons -- and transfer functions).  This is where a neural network
> gets its fault tolerance.  I can take a suitably large network, and change
> a subset of its connection weights to random values without causing its 
> output to leave whatever tolerance I choose.  The smaller the network or
> tighter my tolerance, the smaller the number of weights I can change.
> It's important to remember that any particular memory stored in a neural
> network is distributed through _all_ of the neurons.  Some play a larger
> role than others, to be sure, but the distribution is still complete.

This puzzles me slightly, so please bear with me.  

From what I'd read so far, it appeared to me that the memories -are-
associated with the means used to store them (don't different transfer
functions apply to storage of different types of memory, some transfer
functions being unsuitable for learning in certain cases ?), and if
the distribution -is- complete, as I had understood, that brings me
back to my earlier query about the extraction (i.e. copying) of a
skill or set of skills, presumably associated with a set of memories.
I'm assuming here that all present skills are dependent to a degree
upon skills learned earlier and so cannot be retrieved in isolation.

The original proposal, as I understand (and remember) it, had been to
clone appropriate skills from the primary (e.g. me) into Me2, Me3, and
so on.  My concern was about exactly how it might be possible to
extract those skills in their entirety when it might not even be
possible to identify them, let alone pull them out in sufficient
detail for them to be usable.

> 
> A useful image to keep in mind is the graph of the energy of association
> for various memories.  Imagine a hyperplane in (n+1)-space, where n 
> is the number of weights in the network.  The last axis is the total
> "energy of association" in the network.  Each memory that the network is
> required to learn creates a (generally complex) 'dimple' in the surface.
> The "lowest" point in that dimple represents the optimal set of weights
> that will allow a network to learn that association.  Since we usually
> want a network to learn a large number of associations, the hypersurface
> becomes very warped.  The object of most learning algorithms is to find
> the point on that complex surface that minimizes the energy without
> being trapped in any local minima.

I have great difficulty imagining objects in anything other than three
planes (and since I have a lousy sense of direction, that probably
applies to two planes as well :-)), but I get your point.  The obvious
question which follows on is: how do you decide which dimple or group
of dimples corresponds to a particular skill/memory ?  And if we still
take the parallel dump from clones to primary as a possibility, would
it be possible to differentiate the new skills/memories from the set
supplied to form the clone(s) in the first place ?

Peter "On second-hand thoughts..." Brooks