Edit model card

An eagle soaring above a transformer robot

! Important Note !

The following is the full RWKV-5 Eagle 7B model weights, which can be used with our various inference libraries Download link here

For HF compatible implementation, refer to here

This is not an instruct tune model! (soon...)

Eagle 7B - in short

Eagle 7B is a 7.52B parameter model that:

  • Built on the RWKV-v5 architecture (a linear transformer with 10-100x+ lower inference cost)
  • Ranks as the world’s greenest 7B model (per token)
  • Trained on 1.1 Trillion Tokens across 100+ languages (70% English, 15% multi lang, 15% code)
  • Outperforms all 7B class models in multi-lingual benchmarks
  • Approaches Falcon (1.5T), LLaMA2 (2T), Mistral (>2T?) level of performance in English evals
  • Trade blows with MPT-7B (1T) in English evals
  • All while being an “Attention-Free Transformer”
  • Is a foundation model, with a very small instruct tune - further fine-tuning is required for various use cases!

Find out more at our model announcement: https://blog.rwkv.com/p/eagle-7b-soaring-past-transformers

Or our wiki: https://wiki.rwkv.com

Downloads last month
Unable to determine this model's library. Check the docs .

Space using RWKV/v5-Eagle-7B-pth 1