yzhangcs (Yu Zhang)

upvoted a paper 14 days ago

Simple linear attention language models balance the recall-throughput tradeoff

Paper • 2402.18668 • Published Feb 28 • 17

upvoted a paper 18 days ago

Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models

Paper • 2401.04658 • Published Jan 9 • 24

upvoted an article 22 days ago

Article

Introduction to State Space Models (SSM)

By

•

Apr 8

• 25

upvoted 7 papers about 1 month ago

Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length

Paper • 2404.08801 • Published Apr 12 • 62

Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention

Paper • 2404.07143 • Published Apr 10 • 92

Linear Transformers with Learnable Kernel Functions are Better In-Context Models

Paper • 2402.10644 • Published Feb 16 • 74

Textbooks Are All You Need II: phi-1.5 technical report

Paper • 2309.05463 • Published Sep 11, 2023 • 84

upvoted an article about 1 month ago

Article

Cosmopedia: how to create large-scale synthetic data for pre-training Large Language Models

Mar 20

• 22

upvoted a paper about 2 months ago

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits

Paper • 2402.17764 • Published Feb 27 • 566

upvoted a collection about 2 months ago

based

Collection

These language model checkpoints are trained at the 360M and 1.3Bn parameter scales for up to 50Bn tokens on the Pile corpus, for research purposes. • 14 items • Updated 9 days ago • 8

upvoted 2 papers 2 months ago

Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling

Paper • 2403.03234 • Published Mar 5 • 11

The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry

Paper • 2402.04347 • Published Feb 6 • 12

upvoted 2 papers 3 months ago

Scaling TransNormer to 175 Billion Parameters

Paper • 2307.14995 • Published Jul 27, 2023 • 21

BitDelta: Your Fine-Tune May Only Be Worth One Bit

Paper • 2402.10193 • Published Feb 15 • 17

upvoted a paper 4 months ago

TinyLlama: An Open-Source Small Language Model

Paper • 2401.02385 • Published Jan 4 • 81

upvoted a collection 6 months ago

Training Datasets

Collection

A collection of pseudo-labelled datasets used to train the Distil-Whisper model. • 9 items • Updated Mar 21 • 12

upvoted a paper 7 months ago

LoRAShear: Efficient Large Language Model Structured Pruning and Knowledge Recovery

Paper • 2310.18356 • Published Oct 24, 2023 • 22

upvoted a paper 8 months ago

Language Modeling Is Compression

Paper • 2309.10668 • Published Sep 19, 2023 • 79

Yu Zhang

AI & ML interests

Organizations

yzhangcs's activity