BlinkDL commited on
Commit
e637e53
1 Parent(s): aec24d1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -2
README.md CHANGED
@@ -18,9 +18,11 @@ datasets:
18
 
19
  RWKV-4-pile models finetuning on [RedPajama + some of Pile v2 = 1.7T tokens]. Updated with 2020+2021+2022 data, and better at all European languages.
20
 
21
- These are intermedia checkpoints (XXXGtokens means finetuned for XXXG tokens). They are not instruct tuned yet, and they are recommended to replace vanilla Pile models.
22
 
23
- You can already use them because I am finetuning from Pile models (instead of retraining).
 
 
24
 
25
  See https://github.com/BlinkDL/RWKV-LM for details.
26
 
 
18
 
19
  RWKV-4-pile models finetuning on [RedPajama + some of Pile v2 = 1.7T tokens]. Updated with 2020+2021+2022 data, and better at all European languages.
20
 
21
+ Although some of these are intermedia checkpoints (XXXGtokens means finetuned for XXXG tokens), you can already use them because I am finetuning from Pile models (instead of retraining).
22
 
23
+ Note: not instruct tuned yet, and recommended to replace vanilla Pile models.
24
+
25
+ 7B and 14B coming soon.
26
 
27
  See https://github.com/BlinkDL/RWKV-LM for details.
28