I think i figured out how to build AGI. Want to get some feedback.
Discussion
It is theorized in neuroscience field that human brains work by the free energy principle.

https://en.wikipedia.org/wiki/Free_energy_principle


The free energy principle proposes that biological systems, including the brain, work to minimize "surprise" (or prediction error) between their internal models and their sensory inputs. In essence, organisms try to maintain their state within expected bounds by either:


* Updating their internal models to better match reality (perception)

* Acting to change their environment to match their predictions (action)


Think of it like a thermostat that both predicts room temperature and acts to maintain it within an expected range. This principle suggests that all biological self-organizing systems naturally work to minimize the difference between what they expect and what they experience.


If this theory was true, it seems likely that such a system could be replicated in machine learning field. And turns out, it was successfully implemented, in this reinforcement learning algorithm SMIRL.

SMiRL: Surprise Minimizing Reinforcement Learning in Unstable Environments

https://arxiv.org/abs/1912.05510


Interesting things from this paper:

* This algorithm works without explicitly stating any goals.

* It is great at imitation learning.

* It is a great additional reward signal, when the main reward signal is sparse and rare.

* You would think that the surprise minimizing agent, would not do any kind of exploration. But it actually did. It seems, that curiosity, exploration, naturally emerges from surprise minimization, because even if it increased short term surprise, it decreased the long term surprise considerably.


I then realized, that the way this SMIRL model works, is very similar to how Liquid Time Constant Networks work.

https://arxiv.org/abs/2006.04439

They are similar in a sense, that it would explain WHY Liquid neural networks work at all, as even people who invented it have little clue why it actually works.

Here is the video, of a LTC network driving a car, with just 19 neurons: https://x.com/MIT_CSAIL/status/1316033611368366080

Here is the full video from which that twitter video clip was taken from:

https://youtu.be/IlliqYiRhMU?si=nstNmmU7Nwo06KSJ&t=1971

Closed Form Continuous Time Neural network, is an updated version of liquid neural networks. In its paper, the car driving task is examined.

https://arxiv.org/abs/2106.13898

In comparison, it would have taken 1000s of neurons for other models to do the same task of driving this car.


Remarkable things about it:

* It can achieve the same things as other neural networks, with 10-20x less neurons.

* It somehow learns true causality relationships of the world.

* It has excellent skills of generalizing out of distribution, doing the same task with completely different context.

* It can work without stating any goals.

* It is great at imitation learning.


The new modification that LTC models bring, is that they allow variable speed of change for each neuron, in real time. And that alone, led to all those innovations.


This LCT model was trained using offline backpropagation. But then, i stumbled upon a version of LTC model, that learned in real time, online. Like how actual human brains learn.

"Accurate online training of dynamical spiking neural networks through Forward Propagation Through Time"

https://arxiv.org/abs/2112.11231


This is a combination of Forward Propagation in Time+ Liquid Time Constants+ Spiking Neural Networks.

Some remarkable things about it:

* Spiking Neural Networks, is how our human brains work.

* Addition of LTC fixed many prior problems of SNN training, bringing it to the SOTA level.


That made me interested in how Spiking neural networks worked. I learned that spiking neural networks, is how real human brains work. And the learning is done via spike timing dependent plasticity (STDP). Problem was, that no one prior was able to create an effective algorithm for STDP learning for artificial neural networks.


That might be because STDP learning is actually incredibly diverse, variable. Meaning that the standard model of STDP was insufficient to describe all variations of STDP learning.


"Beyond STDP-towards diverse and functionally relevant plasticity rules"

https://www.researchgate.net/publication/326690440_Beyond_STDP-towards_diverse_and_functionally_relevant_plasticity_rules

That made me stumble upon this research paper.

"Sequence anticipation and spike-timing-dependent plasticity emerge from a predictive learning rule"

https://www.researchgate.net/publication/373262499_Sequence_anticipation_and_spike-timing-dependent_plasticity_emerge_from_a_predictive_learning_rule


What those researchers did, was that they basically made a learning algorithm that tried to minimize its surprise, and to make accurate predictions. And that individual neuron level surprize minimization behavior, led to the emergence of STDP learning. So surprise minimization based learning rule for neural networks, by itself emerged into STDP learning rule. And this learning rule, also was able to create different variations of the STDP learning rule, that matches the diversity of it in the human brain.


So those neuroscience researchers basically discovered an effective learning algorithm for Spiking Neural Networks.


So it truly seems, that surprise minimization, underlies literally everything inside the brain. From the general cognition, to the level of individual neurons.


So what if we combine Liquid Time Constant Neural networks, with this new surprise minimization based learning rule for individual neurons? Here is what i theorize what this model would be like:

* It can learn in real time, without the need for backpropagation. 10-20x lowering the training time and costs.

* Surprise minimization naturally leads to curiousity, exploration, so as to minimize total long term surprise. So this model will naturally conduct self-play, exploration, etc. and be capable of learning without any supervision.

* The SMIRL model was capable of playing videogames by itself. You can create a video game around learning and using language, using an LLM. And this model will be able to master this videogame by itself and learn language that way by itself.

And it would learn language, with 100x less training material, compared to LLMs. Because it already had ability to reason prior. While in LLMs, reasoning emerges only while learning language.


So now you would have an AI, that can continuously learn, improve, and which learned to use the language as a tool. Its cognition, reasoning would would have been there before learning Language, not after. Learning to use language would just enhance its reasoning.


Why would this be AGI? Why would this be better than LLMs? We can find that out, by looking at what LLMs are bad at. LLMs are bad at true learning. They need millions of examples of text about some topic, about some skills, to be good at it. It can't learn things with few examples, for the life of it. This is brilliantly illustrated, by the ARC-AGI benchmark.

https://arcprize.org/


Why are LLMs bad at solving those new problems that are out of distribution from its training data? LLMs are bad at solving ARC-AGI puzzles, because they have no knowledge, of the PROCESS of problem solving, puzzle solving. It doesn't have the mental ROUTINES, habits, that we constantly use for problem solving, and living in general. What do i mean?


It can be explained by this research paper about AI, from 1987.


"1987-Pengi: An Implementation of a Theory of Activity"

https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=cb53a49a1187650196cf10835a0193ae0201a75f

And by this paper from 2007, by Hubert Dreyfus.

"Why Heideggerian AI Failed and how Fixing it would Require making it more Heideggerian"

https://leidlmair.at/doc/WhyHeideggerianAIFailed.pdf


What they basically say:

* Temporal nature of the world (things happening in real time, continuously) and the constant interaction of the humans with it, is critical for the functioning of human intelligence.

* Humans constantly use routines, to function in the world. It allows them to save tons of computational energy.

* Humans use mental routines, when they are achieving goals, solving problems, puzzles.

* You cannot model good intelligence, without a mechanism for formation of routines and their usage.

Why do they say, that good AI cannot be created, if it is not in constant contact with continuous time, real time environment? Because this constant interaction with the environment, allows us to remove the need to make prediction in 95% of cases. It allows you to use much simpler routines, that still achieve highly accurate results. Saving tons of energy, computation, memory. It allows you to remove the need for 95% of memories.

Example:

Lets say you want to dive into a pool. But then realize that it might be very cold.

There are 2 things you can do:

make a prediction about probability of the pool being cold, from the previously known information. Make plans, predictions, and then decide in the spot, to jump into the pool, cannon ball style.

Just put your finger in the pool. If its cold, you would decide to not dive into the pool.

For 95% of the tasks that humans constantly encounter, the second way of achieving goals, of doing tasks, is sufficient. Because truly, if you used your full cognition for literally every single micro-decision you have to make, your brain will just get fried. It simply won't be able to keep up with updating time. By the time you make a prediction, plan, goal, and decide to take an action based on it, the time already went by, and you have 10 more tasks you need to urgently finish.

In this particular instance, the second solution, is a routine for automatic error-correction, self correction. Sure, your finger is now wet. But that is not a tragedy, it is a trivial loss. Yet it allowed you to avoid having to plan, predict, define goals, etc. in this scenario, saving tons of brian energy.

There are 100s of such error-correction, self-correction routines in the human brains, that allow you to avoid having to make predictions, plans, etc. saving tons of brain power and time.


Second example:

You guys probably have PC or laptops. Well, you don't need to plan every day, to sit in front of it. What happens, is that you see your PC, and that activates a habit/routine in your brain, that makes your turn it on and scroll reddit. Planning is unnecessary here, because the environment itself serves as a trigger, for an appropriate action, in appropriate time and place.


Now this makes it more obvious, why LLMs are very problematic for achieving general intelligence. Because they are cut-off from the constant interaction with the world. Making them hugely reliant, on planning, prediction making, goal driven behavior, because it cannot leverage the interaction with the real world, to develop simple routines, to course correct its behavior along the way.

By this analogy, Language models do use their full 100% cognition for every micro decision they have to make. Unlike us humans.


Fun fact - a "disadvantage" of liquid neural networks, is that they can only be trained on temporal, continuous-time data. Like video, audio. And not on text. Constant interaction with the world, is the life and blood of liquid neural networks! It literally cannot function without it. Just like real human cognition.

(To clarify, there are liquid network based language models, so it is possible to find a solution around this problem. But by default liquid networks cannot be trained on non-temporal data.)


What is a routine? Let me give you examples of mental routines we use, when we solve problems, puzzles.

* When you ride a bicycle, do you constantly predict the position of your body, its inertia, etc. based on laws of physics, using formulas, and then after making a prediction, adjust your actions, then make a new prediction, again and again? No, you just ride the bicycle, without awareness of such calculations. Because such calculations are not happening. Such predictions are not happening. What happens in actuality, is that you simply developed routines, for self-correcting your center of mass. When you lean slightly right, more than you should, it simply triggers a routine in your brain that makes you tilt slightly to the opposite side.

* We use the same invisible routines, when we solve problems. Example: When you have an object at hand, you are capable of instantly seeing how far you can throw it, what trajectory it will follow, and where it will roughly land. This is problem solving. Yet, you perform it constantly, without using any kind of physics formulas. Because humans have developed effortless mental routines, for correctly throwing things.


And there are 100s or more such routines we use for problem solving, that we are simply are not aware of, that we can't explicitly write into the AI model. The only way the AI can learn those routines, is by learning those routines by itself.


The LLMs cannot solve ARC-AGI puzzles, that average humans can easily solve, because it has no knowledge about the process of problem solving. Only about its description. Current top LLMs only are able to infer only small amount of implicit hidden mental routines, that humans use for problem solving, from texts available in the internet.

LLMs are good at math and coding, because the problem solving routines for those tasks are explicit, are extensively described in texts. With formulas, etc. There are no textbooks, describing the formulas of implicit routines inside the human brain.


This is where my previously described neural network model comes in.


It is my belief that Liquid Time Constant Networks, work based on routines, just like humans. That is what allows it to perform the same task that would take a traditional neural network 1000s of neurons, in just 19 neurons. Because it doesn't need to make any predictions. It is able to encode just handful or routines in those 19 neurons, that enable it do the same tasks, without a need to make any kind of predictions.


If my proposed neural network is better, surely it could solve an ARC-AGI puzzle then, right? I believe so. Here is how this AI model can solve the ARC-AGI puzzles.

* Record many videos, of people solving ARC-AGI puzzles, solving the public dataset problems.

* Put eye trackers on those people, so that it is visible whey those people are looking at.

* Record the brain scans of the people solving those puzzles. Certain mental routines will activate certain brain regions, in certain sequences, giving the AI more clues for reverse-engineering those routines.

* Train the liquid neural network on this data.

Here is the result i expect:

* The liquid neural network will be able to reverse-engineer the problem solving routines people use, and be able to use it itself.


Then just ask it to solve a new ARC-AGI problem, and it will solve it.


This post is all over the place. But yea, i hope you got the general idea behind this AGI architecture.


Upvote
457

Downvote

255
Go to comments


3

Share
Share
Add a comment
Sort by:

Best

Search Comments
Expand comment search
Comments Section
Someone13574
•
6 小時前
•
Edited 6 小時前
•
Profile Badge for the Achievement Top 10% Commenter Top 10% Commenter
Yeah, you are going to need a lot more experimental evidence than that; you can't just claim that a technique will be multiple orders of magnitude better just by linking some papers which don't show that in any way.

You made some pretty huge leaps in logic from "snn's are how human brains work", "fptt is better than backprop at training snn's", and "suprise minimization approximates STDP" to having a model which learns with 10000x less data and is 100x less expensive/faster to train.

Also, that 19 neuron example isn't exactly correct. It is 19 neurons making the decisions, but there is still a whole cnn model taking in the input and presumably doing the heavy lifting by extracting features.

And neural plasticity probably doesn't matter much for short-term tasks like ARC-AGI, that is just using already existing neural circuits. Essentially it is just the same as in-context learning which LLM's can do, and also doesn't require millions of training examples.

I think that if you actually try implementing this, you will find that back-propagation over a dense network is still much better. If not, then congratulations, you found the magic bullet which thousands of researchers over decades of work didn't find全部翻譯成中文不可省略