"The modern project of creating human-like artificial intelligence (AI) started after World War II, when it was discovered that electronic computers are not just number-crunching machines, but can also manipulate symbols. It is possible to pursue this goal without assuming that machine intelligence is identical to human intelligence. This is known as weak AI."

Eeee, errr, well, no, it's not. Strong AI is the pursuit of human-level (not human-like), general purpose intelligence. Weak AI is the pursuit of solutions to difficult individual problems or classes of problems. Source: my AI classes, ca. 1989.

This article isn't starting well.

Edit: And I'm back after reading it all.

There are two important points when evaluating philosophical arguments about artificial intelligence.

1. Have they introduced dualism? (Usually, it's "how have they snuck in dualism without admitting it." Sometimes it's easy to see, as in Searle's Chinese Room thing. Othertimes not so much.)

2. What happens if you continue with their own questions? Does it lead to an unpalatable (or simply wrong) conclusion?

"The main thesis of this paper is that we will not be able to realize AGI because computers are not in the world."

To start with number 2, who exactly is "in the world"? A blind person? A deaf person? A quadriplegic person? A deaf-blind-mute-quadriplegic-from-birth person? That poor bastard from Johnny Got His Gun? How about this laptop? A robot? A robot that is indistinguishable from a human being without physiological tests? The author started out with an elaborate discussion of human-like and non-human-like general intelligence, but now has unrolled that (Without suitable caution signs. Health and Safety are going to be pissed.) and is now only interested in human-like artificial general intelligence, only in something that resembles a human being: "As Hubert Dreyfus pointed out, we are bodily and social beings, living in a material and social world. To understand another person is not to look into the chemistry of that person’s brain, not even into that person’s “soul”, but is rather to be in that person’s “shoes”. It is to understand the person’s lifeworld." (And I'm going to go out on another limb and ask how well anyone, anywhere, understands some other persons' lifeworld. I don't think that is possible.)

"However, there is a problem with both these quotations. If Harari and Crick are right, then the quotations are “nothing but” the result of chemical algorithms and “no more than” the behavior of a vast assembly of nerve cells. How can they then be true?"

And here we have number 1. If materialism is right, how can those quotations be true? How can anything said or done by humans be true? They can't be; they're just biochemical algorithms, neurons, and molecules. Truth requires a soul, and a soul could completely grok some other person's lifeworld. Obviously.

So clearly, nothing even remotely like Watson or AlphaGo would be intelligent. (Even if they're not intended to be general intelligences. Yes, he spent the last part of the article complaining that two systems weren't something that they were never intended to be, in spite of spending a large time at the beginning delineating that very difference.)