GoldFinch: High Performance RWKV/Transformer Hybrid with Linear Pre-Fill and Extreme KV-Cache Compression Paper โข 2407.12077 โข Published Jul 16, 2024 โข 57
Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence Paper โข 2404.05892 โข Published Apr 8, 2024 โข 39
The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits Paper โข 2402.17764 โข Published Feb 27, 2024 โข 620
Watermarking Makes Language Models Radioactive Paper โข 2402.14904 โข Published Feb 22, 2024 โข 25
LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens Paper โข 2402.13753 โข Published Feb 21, 2024 โข 117
BlackMamba: Mixture of Experts for State-Space Models Paper โข 2402.01771 โข Published Feb 1, 2024 โข 26