(Former emerging tech consultant for ~10% of Fortune 500 here)

(a) I've noticed a common trend of AI researchers looking at the tree in front of them and saying "well, this tree is not also a forest and won't be any time soon."

But there's not always awareness of what's going on in other specialized domains, so an AI vision researcher might not be intimately aware of what's currently being done in text or in "machine scientists" in biology for example.

As well, it overlooks the development of specialization of the human brain. We have some specialized structures that figured their niche out back with lizards, and others that developed much later on. And each of those specialized functions work together to give rise to 'human' intelligence.

So GPT-3 might be the equivalent of something like the Wernicke's area, and yes - on its own it's just a specialized tool. But what happens as these specialized tools start interconnecting?

Throw GPT-3 together with Dall-E 2 and the set of use cases is greater than just the sum of the parts.

This is going to continue to occur as long as specialized systems continue to improve and emerge.

And quickly we'll be moving into territory where orchestration of those connections is a niche that we'll both have data on (from human usage/selection of the specialist parts) and will in turn build meta-models to automate sub-specialized models from that data.

Deterministic reasoning seems like a niche where a GAN approach will still find a place. As long as we have a way for one specialized model to identify "are these steps leading to X" we can have other models only concerned with "generate steps predicted to lead to X."

I don't think we'll see a single model that does it all, because there's absolutely no generalized intelligence in nature that isn't built upon specialized parts anyways, and I'd be surprised if nature optimized excessively inefficiently in that progress.

Will this truly be AGI in a self-determining way? Well, it will at least get closer and closer to it with each iteration, and because of the nature of interconnected solutions, will probably have a compounding rate of growth.

In a theoretical "consciousness" sense of AGI, I think the integrated information theory is interesting, and there was a paper a few years ago about how there's not enough self-interaction of information possible in classical computing to give rise to consciousness, but we'll probably have photonics in commercial grade AI setups within two years, so as hand-wavy as the IIT theory is, the medium will be shifting towards one compatible with their view of consciousness-capable infrastructure much sooner for AI than quantum competing in general.

So I'd guess we may see AI that we're effectively unable to determine if it is "generally intelligent" or 'alive' within 10-25 years, though I will acknowledge that AI is the rare emerging tech that I've been consistently wrong about the timing on in a conservative direction (it keeps hitting benchmark improvements faster than I think it will).

(b) The notion AGI will have it out for us is one of the dumbest stances and my personal pet peeves out there, arguably ranked along with the hubris of "a computer will never be able to capture the je ne sais quoi of humanity."

The hands down largest market segment for AI is going to be personalization, from outsourcing our work to a digital twin of ourselves to content curation specific to our own interests and past interactions.

Within a decade, no one is going to give the slightest bit of a crap about interactions with other humans in a Metaverse over interacting with AIs convincingly human enough but with the key difference of actually listening to our BS rather than just waiting for their turn to talk.

There's a decent chance we're even going to see a sizable market for feeding social media data of deceased loved ones and pets into AI to make twins available in such settings (and Microsoft already holds a patent on that).

So do we really think humans are so repugnant that the AI which will eventually reach general intelligence within the context of replicating itself as ourselves, as our closet friends and confidants, as our deceased loved ones - will suddenly decide to wipe us out? And for what gains? What is AI going to selfishly care about land ownership and utilization for?

No. Even if some evolved AGI somehow has access to DARPA killer drones and Musk's Terminator robots and Boston Dynamics' creepy dogs, I would suspect a much likelier target would be specific individuals responsible for mass human suffering the AIs will be exposed to (pedophiles, drug kingpins, tyrants) than it is grandma and little Timmy.

We're designing AI to mirror us. The same way some of the current thinking of how empathy arises in humans is from our mirror neurons and the ability to put ourselves in the shoes of another, I'm deeply skeptical of the notion that AI which we are going to be intimately having step into human shoes will become some alien psychopath.