Newsgroups: bit.listserv.csg-l
Date:         Fri, 6 Nov 1992 11:01:41 EST
From:         mmt@BEN.DCIEM.DND.CA
Subject: Re: CONNECTIONISM AND hpct

[Martin Taylor 921106 10:45]
(Bill Powers 921104.2100)

> The difference you see depends on the dimension you're attending to.
> For me, the difference between AI and connectionism is that
> connectionist models let the signals representing variables be the
> important thing, where in AI all variables had to be converted first
> into symbols (words, mainly) before they could be operated upon, and
> then the operations were carried out by rule-driven algorithms for
> symbol manipulation instead of by computing devices that handle
> signals directly. To me, that's the difference between analogue and
> digital computing.

Fair enough.  There are certainly two dimensions (at least) in the comparison.
But it appears to me that HPCT allows algorithmic manipulations at the
program level and higher (possibly as low as the category level), so the
analogue-discrete distinction doesn't fully distinguish PCT from AI.

> "It is convenient to think of the brain as a collection of localized
> functions, and of neural signals as occurring in definite pathways
> linking functions together. The model, however, will not be
> invalidated if these elements prove some day to be distributed over
> large volumes of the brain. The organizational properties of this
> model do not depend on its geometrical properties." BCP, p. 39

I have a new copy of BCP in its plastic wrapping on my desk, but I haven't
read it for over a year now, so I didn't remember this paragraph.  However,
I would argue that one could go further: not only is the model not invalidated,
it is enhanced by distribution.  Indeed, I think distribution is the only way
that a full-blooded PCT system could work in a real, as opposed to a modelled
world.  A non-distributed (localized, near-orthogonal ECS) system will be as
brittle as a rule-based AI system (perhaps I exaggerate) as compared to a
neural network performing a similar task.

> The connectionist models I have seen do not impress me as much as
> they impress their inventors. A great deal of subjective
> interpretation is involved in saying that a network "classifies" its
> inputs, or even that it "recognizes" a form. What these networks
> actually do is a lot simpler than that: given a set of inputs, they
> will produce certain outputs over a range of the inputs. To label
> this process "classification" implies first that one already knows what
> classification is, and second that there is no other kind of
> perceptual operation of any importance. I reject both implications as
> unwarranted, the first because there has been no careful
> investigation of the elements of perception (of the kind I have tried
> to develop) and the second because it is obvious, at least to me,
> that a great deal more than classification goes on in perception. I
> think that connectionists are trying to accomplish in one jump what
> the real perceptual system does stage by stage. This may be possible,
> to some extent, but this sort of modeling will necessarily fail to
> account for perceptions of both lower and higher levels than
> classifications: perception of motion, for example, or perception of
> principles.

Here, I fully agree.  But the performance of the networks on the tasks set
them is impressive to more than their inventors.  It impresses me.  What I
think (and thought for many years before I heard of PCT--I did my first try
at neural nets in the early 60s) is that neural networks are totally misused
when they are limited to classification.  They are good at that (by which I
mean the discrimination among predetermined complex patterns in sapce and time)
but they are better as transformers and coordinators.  I have for a long time
thought that the appropriate use of NN technology was to provide ways of
linking stimuli to behaviour, and since learning of PCT it seems to me clear
how this can be done effectively.

I think I'll leave to another posting with a different focus the importance of
the "squashing function" in the perceptual input path.  It came up last year,
and was apparently misunderstood.  It relates directly to the question of
distribution, but needs some technical preparation.

Martin