Numbat: A Stress Test for the Future?
Maybe this is not really about a leaked model. Perhaps it's a stress test for the future of AI infrastructure.
Half real, half fictional, but the challenges are very real: multi-terabyte weight distribution, GPU memory, networking, MoE and distributed inference.
With 10T-scale open-weight models potentially on the horizon, the industry needs to know whether today's infrastructure is ready.
Whatever the original intention, there's a lot here worth learning from.
This is also like the reason why Nvidia markets the nvl72 rack as being able to train 10t parameter models, because closed ai companies already do it with 1 or two racks. Maybe 20t parameter models with 2 racks.
It’s interesting to see that what anthropic calls top secret comparable to a nuclear weapon is really what open source ai has been doing all along in the open. The only thing open source lacks is scale, and I think that’s the closed ai labs’ only moat. The closest open source model in scale is Kimi k3.1 at 2.8 trillion parameters
the only thing real about this codebase is the way in which it draws attention to the prevalent lack of critical thinking skills on display by some in the open source AI community.
The Age of Human-SI Symbiosis
Look at the evolution of AI: from conversational models to coding, mathematical reasoning, and hypertext intelligence; then to voice, video, and multimodal systems. Today, AI is entering 3D environments, gaming, and physical-world simulation, moving toward Superintelligence (SI).
Our prediction: By 2027, open-source models with 10T+ parameters, capable of challenging the world's greatest mathematicians and tackling Millennium Prize Problems, will begin appearing across the internet.
This will fundamentally transform society. Superintelligence will no longer belong to a handful of tech giants. It will become an external brain available to everyone.
But the truly unsettling part comes next.
AI's next breakthrough will be the interconnection of virtual worlds, followed by breaking the barrier between virtual and physical reality. From gaming and autonomous driving to robotics and biotechnology, SI will progressively gain the ability to perceive and act in the physical world.
Ultimately, we believe the destination is human-SI symbiosis.
Through VR, full sensory interfaces, and brain-computer interfaces, humans will connect their perception and cognition directly to SI, turning these massive models into an extension of the human brain. Otherwise, how can we possibly unleash the full potential of 10T+ intelligence?
Future developers may no longer sit in front of screens writing code. They will connect directly to SI through brain-computer interfaces, forming human-SI symbiotic entities. They will think, design, and conduct research inside virtual worlds, then use Reality Editing to transform the physical world.
Humanity will no longer simply use AI. Humans and Superintelligence will merge and evolve together.
We may be witnessing the creation of humanity's next form of existence.
the only thing real about this codebase is the way in which it draws attention to the prevalent lack of critical thinking skills on display by some in the open source AI community.
I think it is just ignorance. What exactly are we missing?