BlinkDL commited on
Commit
2a46544
1 Parent(s): a147141

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -1
README.md CHANGED
@@ -22,4 +22,9 @@ At this moment you have to use my Github code (https://github.com/BlinkDL/RWKV-v
22
 
23
  ctx_len = 768 n_layer = 24 n_embd = 1024
24
 
25
- Final checkpoint: 20220615-10803.pth : Trained on the Pile for 331B tokens. Pile loss 2.349. LAMBADA ppl 15.34, acc 42.42%. PIQA acc 67.03%. SC2016 acc 62.05%. Hellaswag acc_norm 38.47%.
 
 
 
 
 
 
22
 
23
  ctx_len = 768 n_layer = 24 n_embd = 1024
24
 
25
+ Final checkpoint: 20220615-10803.pth : Trained on the Pile for 331B tokens.
26
+ * Pile loss 2.349
27
+ * LAMBADA ppl 15.34, acc 42.42%
28
+ * PIQA acc 67.03%
29
+ * SC2016 acc 62.05%
30
+ * Hellaswag acc_norm 38.47%