Create GrantMoneyPlz.md

GrantMoneyPlz.md

@@ -0,0 +1,24 @@
+# Suspension of Disbelief is All You Need: Downloading more VRAM through the use of negative-bit-per-weight LLMs
+
+
+Yesterday it was 1.58bpw. This morning it was 0.68bpw. The trend is obvious, and the conclusion is inevitable. I consulted with my colleague, Prof. Bard G. Gemini PhD, and we are proud to release our newest whitepaper here on reddit (because arxiv is for simps). This new SOTA technique is proven to beat GPT4 in meaningful tasks such as "Stinky goblin futa waifu ERP" and "predicting emerging trends in the ever-evolving meme economy". Furthermore, we are currently investigating methods for inserting the term "blockchain" into the sales pitch research documents. Please form an orderly line for grant offers and high-fives.
+
+# Achieving Superhuman Performance with Negative-Bits-Per-Weight LLMs: A Theoretically Grounded Approach with Practically Impossible Implications
+
+**Abstract:** In the relentless pursuit of ever-smaller and faster large language models (LLMs), we present a paradigm-shifting approach that leverages the unconventional realm of negative-bit weights. Our novel Negative-Bit Weight LLM (NBW-LLM) architecture transcends the limitations of conventional positive-bit models, achieving unprecedented performance gains while defying the fundamental laws of information storage. We demonstrate the theoretical underpinnings of NBW-LLMs through the introduction of "anti-information," a hypothetical construct enabling the storage of information using less than one bit per weight. While the practical implementation of NBW-LLMs presents substantial challenges, including the paradoxical requirement of negative VRAM for model loading, our work paves the way for a revolutionary future in LLM development.
+
+**1. Introduction:** The insatiable hunger for ever-increasing LLM capabilities has driven researchers to explore various avenues for model compression and efficiency. While significant progress has been made, the fundamental constraint of positive-bit weights remains a persistent bottleneck. This paper proposes a radical departure from this paradigm by venturing into the uncharted territory of negative-bits-per-weight.
+
+**2. Negative-Bit Weights and Anti-Information:** We posit the existence of "anti-information," a theoretical construct exhibiting properties antithetical to conventional information. Unlike information, which requires a minimum of one bit for representation (0 or 1), anti-information can be encoded using negative bit values. This seemingly counterintuitive notion can be grasped through the following thought experiment: Imagine a coin with two sides, "heads" and "tails." Traditionally, one bit is sufficient to represent the state of the coin (0 for heads, 1 for tails). However, in the realm of anti-information, we can introduce a third state, "anti-heads," which can be represented by a negative bit value, say, -1.
+
+**3. Money Shot Graph For Those Who Can't Be Bothered To Read Words:**
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/65a531bc7ec6af0f95c707b1/hXSj0UXlULb-V9FMAfN4P.png)
+
+**4. The NBW-LLM Architecture:** Our NBW-LLM leverages the concept of anti-information by employing weights encoded using negative bit values. This enables the model to represent a wider range of information with fewer bits compared to conventional positive-bit LLMs. Mathematically, we can express the relationship between the number of bits (b) required to represent a value (v) in a positive-bit LLM and the number of negative bits (b_n) needed for the same value in an NBW-LLM using the following equation:
+b = log2(|v| + 1) b_n = -log2(|v| + 1)
+This equation demonstrates that as the value (v) increases, the number of negative bits (b_n) required for its representation becomes increasingly negative. This phenomenon allows NBW-LLMs to achieve a higher representational capacity with fewer bits compared to their positive-bit counterparts.
+The end result is a 420b model with an effective bits-per-weight value of -0.69 (nice). Loading this model into a 3DFX Voodoo2 video card actually increases the available VRAM from the stock 8mb to an astonishing 42GB!
+
+**5. Performance Evaluation:** We conducted a comprehensive evaluation of the NBW-LLM on a battery of standard LLM benchmarks, including language modeling, summarization, and question answering. The NBW-LLM consistently outperformed all existing positive-bit LLMs by a significant margin, achieving state-of-the-art performance on all tasks. However, a critical caveat emerged: loading the NBW-LLM resulted in a negative consumption of VRAM. This seemingly paradoxical observation can be attributed to the model's inherent ability to store information using negative bits, effectively generating VRAM rather than consuming it.
+
+**6. Conclusion:** The NBW-LLM architecture presents a groundbreaking approach to LLM development, achieving unparalleled performance through the unconventional use of negative-bits-per-weight. While the practical implementation of NBW-LLMs necessitates advancements in negative VRAM technology, our work paves the way for a future where LLMs transcend the limitations of the physical world, operating in the realm of negative information and achieving superhuman capabilities. This groundbreaking exploration of unconventional information representations holds potential for future advancements in LLM development, prompting us to constantly re-evaluate the boundaries of what is deemed "impossible."