27 84 1

Lee Gao

leegao19

AI & ML interests

None yet

Recent Activity

commented on a paper 18 days ago

Entropy-Guided Attention for Private LLMs

upvoted a paper 21 days ago

Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Though

upvoted a paper 21 days ago

Entropy-Guided Attention for Private LLMs

View all activity

Organizations

leegao19's activity

commented a paper 18 days ago

Entropy-Guided Attention for Private LLMs

Paper • 2501.03489 • Published 27 days ago • 14 •

upvoted 2 papers 21 days ago

Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Though

Paper • 2501.04682 • Published 25 days ago • 90

Entropy-Guided Attention for Private LLMs

Paper • 2501.03489 • Published 27 days ago • 14

commented a paper 21 days ago

Entropy-Guided Attention for Private LLMs

Paper • 2501.03489 • Published 27 days ago • 14 •

commented 2 papers 29 days ago

Fourier Position Embedding: Enhancing Attention's Periodic Extension for Length Generalization

Paper • 2412.17739 • Published Dec 23, 2024 • 40 •

Fourier Position Embedding: Enhancing Attention's Periodic Extension for Length Generalization

Paper • 2412.17739 • Published Dec 23, 2024 • 40 •

upvoted a paper 30 days ago

Position Information Emerges in Causal Transformers Without Positional Encodings via Similarity of Nearby Embeddings

Paper • 2501.00073 • Published Dec 30, 2024 • 1

commented a paper 30 days ago

Breaking the Stage Barrier: A Novel Single-Stage Approach to Long Context Extension for Large Language Models

Paper • 2412.07171 • Published Dec 10, 2024 • 1 •

updated 2 collections 30 days ago

Papers I Like

Collection

24 items • Updated 30 days ago

Lee's RoPE Tricks / Context Extension Reads

Collection

Set of Long Context (RoPE or otherwise) I'm collecting off of HF • 45 items • Updated 30 days ago • 3

upvoted a paper 30 days ago

Breaking the Stage Barrier: A Novel Single-Stage Approach to Long Context Extension for Large Language Models

Paper • 2412.07171 • Published Dec 10, 2024 • 1

commented a paper about 1 month ago

Rethinking Addressing in Language Models via Contexualized Equivariant Positional Encoding

Paper • 2501.00712 • Published Jan 1 • 6 •

upvoted a paper about 1 month ago

Rethinking Addressing in Language Models via Contexualized Equivariant Positional Encoding

Paper • 2501.00712 • Published Jan 1 • 6

updated 3 collections about 1 month ago

upvoted a paper about 1 month ago

Understanding and Mitigating Bottlenecks of State Space Models through the Lens of Recency and Over-smoothing

Paper • 2501.00658 • Published Dec 31, 2024 • 7

updated a collection about 1 month ago

Papers I Like

Collection

24 items • Updated 30 days ago

upvoted a paper about 1 month ago

LearnLM: Improving Gemini for Learning

Paper • 2412.16429 • Published Dec 21, 2024 • 22