picocreator commited on
Commit
d322f21
1 Parent(s): be042cd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +19 -0
README.md CHANGED
@@ -1,3 +1,22 @@
1
  ---
2
  license: apache-2.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  ---
4
+
5
+ ![An eagle soaring above a transformer robot](https://substackcdn.com/image/fetch/w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6bbd31a7-21b4-4ff6-b43f-8735d1decf25_2048x1652.png)
6
+
7
+ # Eagle 7B - in short
8
+
9
+ Eagle 7B is a 7.52B parameter model that:
10
+
11
+ - Built on the RWKV-v5 architecture
12
+ (a linear transformer with 10-100x+ lower inference cost)
13
+ - Ranks as the world’s greenest 7B model (per token)
14
+ - Trained on 1.1 Trillion Tokens across 100+ languages
15
+ - Outperforms all 7B class models in multi-lingual benchmarks
16
+ - Approaches Falcon (1.5T), LLaMA2 (2T), Mistral (>2T?) level of performance in English evals
17
+ - Trade blows with MPT-7B (1T) in English evals
18
+ - All while being an “Attention-Free Transformer”
19
+ - Is a foundation model, with a very small instruct tune - further fine-tuning is required for various use cases!
20
+
21
+ Find out more at our model announcment: https://blog.rwkv.com/p/eagle-7b-soaring-past-transformers
22
+ Or our wiki: https://wiki.rwkv.com