Mamba distilled from Zephyr. The Mamba in the Llama: Distilling and Accelerating Hybrid Models (https://arxiv.org/abs/2408.15237).
Junxiong Wang
JunxiongWang
AI & ML interests
Attention Free Model / Subquadratic Language Models
Organizations
Collections
4
models
6
JunxiongWang/MambaByte_Stories
Text Generation
•
Updated
•
30
JunxiongWang/MambaByte_Arxiv
Text Generation
•
Updated
•
15
•
3
JunxiongWang/MambaByte_PG19_353M
Text Generation
•
Updated
•
8
JunxiongWang/MambaByte_Books
Text Generation
•
Updated
•
32
•
1
JunxiongWang/MambaByte_Code
Text Generation
•
Updated
•
408
•
2
JunxiongWang/MambaByte_PG19_972M
Text Generation
•
Updated
•
20