From: axs@cs.bham.ac.uk (Aaron Sloman)
Newsgroups: comp.ai,comp.ai.neural-nets,comp.ai.philosophy,sci.psychology
Subject: Re: Human intelligence vs. Machine intelligence
Summary: Critique of Penrose continues
Date: 11 Oct 92 15:00:41 GMT
Organization: School of Computer Science, University of Birmingham, UK

burt@aupair.cs.athabascau.ca (Burt Voorhees)
    Faculty of Science, Athabasca University
writes:

> Date: 6 Oct 92 19:01:49 GMT
> .....
>   Like the 300 lb gorilla, Penrose can be counted an expert in whatever
> field he chooses to put his mind to.

An entertaining analogy.

>  ......
>
(from Chris Malcolm    cam@uk.ac.ed.aifh)
> >>An indication of the status of Penrose's critique of artificial
> >>intelligence is that -- apart from reviews at the time of publication
> >>-- nobody in the field is bothering to refute it,

No so, see below

> ..whereas the
> >>criticisms of such people as Searle and Dreyfus (which are covered in
> >>the books I cited) have spawned more books and papers than you could
> >>carry.
>
>   More likely, nobody in the AI community can refute them.

Well, I've tried, see below.

>   What seems to be the case in this debate is that people on either side
> are really arguing from hidden metaphysical assumptions and nobody (except
> for the occasional person like Terry Winograd) is going to change sides
> no matter what.  In terms of a research paper this gives one the opportunity
> to investigate such things as the nature of consciousness, the various
> versions of mathematics (e.g., Platonism, formalism, etc: most of the strong
> AI people are formalists, their opponents are often Platonists), and etc.

For information - there are thirty seven short commentaries on
Penrose's book in the December 1990 issue of the "Behavioral and
Brain Sciences" journal. They are preceded by a precis of the book
by Penrose himself, and followed by a collection of (totally
unrepentant) author's replies to the articles.

Chris Malcolm's comment
> >>-- nobody in the field is bothering to refute it

may be based on the perception that most people working in AI feel
that the early reviews adequately demolished Penrose's position.
My own opinion, after reading the book, was that there was quite a
lot more to be said, both:
(a) as to why the published criticisms of Penrose were partially or
wholly unsuccessful (either because of technical errors or omissions
or because they attacked a subtly different position from his)
and
(b) as to why Penrose really was mistaken concerning a number of
issues.

So I wrote a much longer review which, after very detailed criticism
and scrutiny by a collection of friends and colleagues and the review
editors of the AI journal, finally appeared in the AI Journal this
August. The abstract, minus umlauts, etc. is:

   \fIThe Emperor's New Mind\fR by Roger Penrose has received a
   great deal of both praise and criticism. This review discusses
   philosophical aspects of the book that form an attack on the
   ``strong'' AI thesis. Eight different versions of this thesis are
   distinguished, and sources of ambiguity diagnosed, including
   different requirements for relationships between program and
   behaviour.

[When there's so much unacknowledged ambiguity in a thesis, a lot of
the argument about it can be expected to be at cross-purposes.]

       Excessively strong versions attacked by Penrose (and
   Searle) are not worth defending or attacking, whereas weaker
   versions remain problematic. Penrose (like Searle) regards the
   notion of an algorithm as central to AI, whereas it is argued
   here that for the purpose of explaining mental capabilities the
   architecture of an intelligent system is more important than the
   concept of an algorithm, using the premise that what makes
   something intelligent is not what it does but how it does it.
   What needs to be explained is also unclear: Penrose thinks we all
   know what consciousness is and claims that the ability to judge
   Godel's formula to be true depends on it. He also suggests that
   quantum phenomena underly consciousness. This is rebutted by
   arguing that our existing concept of ``consciousness'' is too
   vague and muddled to be of use in science. This and related
   concepts will gradually be replaced by a more powerful
   theory-based taxonomy of types of mental states and processes.

[Just as our descriptive apparatus for describing kinds of stuff was
significantly enhanced by a "generative" theory of the underlying
mechanisms and atomic structures, etc.]

   The central argument offered by Penrose against the strong AI
   thesis depends on a tempting but unjustified interpretation of
   Godel's incompleteness theorem. Some critics are shown to have
   missed the point of his argument. A stronger criticism is
   mounted, and the relevance of mathematical Platonism analysed.
   Architectural requirements for intelligence are discussed and
   differences between serial and parallel implementations analysed.

[Incidentally, on that last point, I was not discussing mathematical
differences but engineering differences: I take AI as a study of the
requirements for *working* intelligent systems, whether natural or
artificial.  From that point of view, issues of complexity,
reliability, etc. are more important than issues of computability.]

The full reference for the review is
  Aaron Sloman `The emperor's real mind: review of Roger Penrose's
  The Emperor's new Mind: Concerning Computers Minds and the Laws of
  Physics,'
  in Artificial Intelligence, 56 (1992) pp 355-396

[An earlier version was circulated quite widely in mid 1991. The
published version is rather different in detail, thanks to fruitful
criticisms by Steve Smoliar, Mark Stefik and others.]

I fear that too many people took Penrose as being inherently hostile
to AI, whereas my impression is that he was misled by some of the more
extreme statements made by some AI protagonists, and since talking to
him I've concluded that he is not hostile to AI as it is actually
practiced, as opposed to AI as it is sometimes described or defined.
Moreover he is genuinely puzzled as to how computational mechanisms
could explain some of his own experiences as a working mathematician.

This is a field in which people with strong prejudices don't pay
sufficiently close attention to what people actually say and do, and
jump to conclusions on the basis of a quick reading. I think Penrose
did that, and some of his critics did the same, the result being
muddle on both sides.

In favour of Penrose I would say that he has raised a real problem
about how the techniques of AI could be used to explain what appear to
be the experiences of mathematicians thinking about the properties of
infinite sets. (Godel's reasoning is an example of this.) My
impression is (a) that there's a collection of problems here that
nobody in AI has addressed adequately and (b) that the experiences of
mathematicians who think that they are reasoning about uniquely
identified infinite sets may be misleading: they may actually be
unwittingly deluding themselves - including both Penrose and some of
his critics. (This is related to the question whethere there is some
version  of mathematical platonism that is true.)

Incidentally, I am still unsure about some of the technical points in
my review, even though they have been "vetted" by several
mathematically and philosophically oriented computer scientists (and
Penrose), so I welcome any indications of flaws in my reasoning, etc.

Aaron
PS
Penrose is writing a new book on these topics. It will be interesting
to see whether he has learnt anything from the criticisms and
discussions following The Emperor's New Mind. I don't know when the
new book will be published.
-- 
Aaron Sloman, School of Computer Science,
The University of Birmingham, B15 2TT, England
EMAIL   A.Sloman@cs.bham.ac.uk  OR A.Sloman@bham.ac.uk
Phone: +44-(0)21-414-3711       Fax:   +44-(0)21-414-4281
Newsgroups: comp.ai,comp.ai.shells