Fix QK norm to apply before RoPE, retrain released checkpoints

by tommiekerssies - opened 25 days ago

base: refs/heads/main

←

from: refs/pr/2

Discussion Files changed

-4

tommiekerssies

Amazon Frontier AI & Robotics org 25 days ago

No description provided.

Fix QK norm to apply before RoPE, release more resolution-flexible checkpoints67539ca9

tommiekerssies changed pull request title from Fix QK norm to apply before RoPE, release more resolution-flexible checkpoints to Fix QK norm to apply before RoPE, retrain released checkpoints 25 days ago

tommiekerssies changed pull request status to merged 25 days ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment