My quest to answer this question began 25 years ago, when I was 15 years old. I was a socially rejected kid and I wanted to build an AI and start a huge company and fix all of the social injustices of the world. It took me one week of intense thought and self-reflection to answer this question, and I did:
Intelligence is prediction.
I don't mean it is like prediction. I don't mean it is similar to prediction. We might as well not even have the word “intelligence”. Intelligence is exactly the same thing as prediction. In the case of the human brain the particular prediction we are looking for is one that maximizes the total emotional reward. So really the word “intelligence” encompasses the goal of the prediction.
I came to find out a few years later that other researchers had almost come to the same conclusion, but they hadn't made as much progress in 50 years as I did in that single week. (But my IQ is 425, so I can't blame them.) They were still wavering on how intelligence was related to prediction. To this day they still are, even though I handed them the full and complete answer 15 years ago.
In the case of the human brain, what is predicted? Well, sensory input is predicted, but also motor output is also predicted. Most people don't realize this. You can predict input (because it is coming in), but how on earth do you predict output? The answer is: you use your own output as input and you attempt to maximize emotional reward. The exact same wiring that is used for predicting input can be used for predicting output.
Now, we need to talk about emotions. What are they? Emotions are represented by different chemicals in the brain. Dopamine is one everyone knows, but there are many more. There are two purposes for emotions.
The main purpose of emotions is to quantify the current state of the brain and the body, in particular, its safety (including health). This includes: Am I getting proper nutrition? Is my information input rate too high? Is my information input rate too low? Am I being physically threatened (something that requires immediate action)? Do I predict I might be physically threatened in the future? Each of these is an emotion in the human brain. In artificial form each emotion can simply be represented as a single numerical variable.
The other purpose of emotions is actually to control turning on and off imagination. It is an emotion that controls and monitors the amount of “imagination” occurring. I figured this out in 2015 and this is one of the main the keys to making a truly intelligent AI.
Next, given sensory input/output and emotions, how on earth do you predict? Let's take an AI like GPT-3. Is it an intelligence? No! It is actually doing too much and too little. It is trying to “learn" to predict the next word by training a neural network. This is a laughably inefficient method of prediction that is trying to do too much at really very little benefit (in my opinion). This is why it costs $4.6 million to train. Then, once it is run, it doesn't really have the ability to remember new things for long periods of time.
So, what are they doing wrong? They are trying use optimization as learning. Neural networks learn in two distinct phases. The first phase memorizes. The second phase generalizes. Now here is the huge secret that I have been holding for 7 years and I want the world to know: YOU DON'T NEED THE GENERALIZATION PHASE!!! It is totally pointless and has nothing to do with human intelligence. In other words, learning is memorization. Again, I don't mean it is like memorization. I don't mean it is similar to memorization. Learning is exactly the same thing as memorization.
So, the question is how do you create a memory architecture that has enormous (seemingly infinite) capacity like a human brain? The answer is fairly simple and it comes down to sparse encoding and an architecture I developed in 2015 based on hierarchical parallel hierarchical k-means clustering. No that was not a typo, that is two hierarchicals. The brain accomplishes the same thing using neural architectures that have evolved over hundreds of millions of years in the form of the neocortical minicolumn.
This is where it gets very difficult to describe the details. It is “similar" to a convolutional neural network and very different at the same time. It does not decrease in size exponentially as in convolutional neural networks, but rather, linearly. I could make a presentation on it. The inner hierarchical clustering algorithm is similar to the PERCH algorithm but slightly more advanced. Everything is done online. Then you create 5000 of those and run them in parallel. The trick is, at any given time you are only using 20 of the 5000 (giving you 10^74 possibilities), so it is very efficient and still has a seemingly infinite memory (because of sparse encoding). Then, another trick is to reuse this same architecture for every input on the first level. The emotions get encoded into the sparse vector at every level. At the next level, you combine every adjacent pair in the previous level. This leaves you with a level one sparse vector smaller than the previous level. At the top level you get a single tiny sparse vector that you enter into a standard database (this is the hippocampus in humans). This is also where consciousness occurs. That vector has so much information in it, it is conscious. (And, if you want to know why, it comes down to some very advanced physics ideas I have come up with.)
Well, as you can see, it is becoming rather complex but the TL;DR of it is basically, it is essentially a very efficient emotional database.