Tri Dao

tridao's activity

• All
• Models
• Datasets
• Spaces
• Papers
• Collections
• Community
• Posts
• Upvotes
• Likes
• Articles

published an article 4 months ago

view article

Article

Bamba: Inference-Efficient Hybrid Mamba2 Model

By and 28 others •

Dec 18, 2024

• 50

authored a paper 5 months ago

RedPajama: an Open Dataset for Training Large Language Models

Paper • 2411.12372 • Published Nov 19, 2024 • 55

authored a paper 7 months ago

The Mamba in the Llama: Distilling and Accelerating Hybrid Models

Paper • 2408.15237 • Published Aug 27, 2024 • 41

updated a collection 10 months ago

Mamba-2

Collection

0 items • Updated Jun 3, 2024

authored 4 papers about 1 year ago

Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling

Paper • 2403.03234 • Published Mar 5, 2024 • 14

StarCoder 2 and The Stack v2: The Next Generation

Paper • 2402.19173 • Published Feb 29, 2024 • 141

BitDelta: Your Fine-Tune May Only Be Worth One Bit

Paper • 2402.10193 • Published Feb 15, 2024 • 22

Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads

Paper • 2401.10774 • Published Jan 19, 2024 • 55

authored 8 papers over 1 year ago

Mamba: Linear-Time Sequence Modeling with Selective State Spaces

Paper • 2312.00752 • Published Dec 1, 2023 • 143

Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time

Paper • 2310.17157 • Published Oct 26, 2023 • 14

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

Paper • 2205.14135 • Published May 27, 2022 • 13

Hungry Hungry Hippos: Towards Language Modeling with State Space Models

Paper • 2212.14052 • Published Dec 28, 2022 • 1

Hyena Hierarchy: Towards Larger Convolutional Language Models

Paper • 2302.10866 • Published Feb 21, 2023 • 7

Simple Hardware-Efficient Long Convolutions for Sequence Modeling

Paper • 2302.06646 • Published Feb 13, 2023 • 2

StarCoder: may the source be with you!

Paper • 2305.06161 • Published May 9, 2023 • 31

FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning

Paper • 2307.08691 • Published Jul 17, 2023 • 8