When Precision Meets Position: BFloat16 Breaks Down RoPE in Long-Context Training
Paper
•
2411.13476
•
Published
•
15
Seongjin Lee
GirinMan
AI & ML interests
NLP, LLM, Langchain
Recent Activity
liked
a model
10 days ago
AIDC-AI/Marco-o1
upvoted
a
paper
about 1 month ago
When Precision Meets Position: BFloat16 Breaks Down RoPE in Long-Context
Training
liked
a model
about 2 months ago
genmo/mochi-1-preview
Organizations
