Update README.md
Browse files
README.md
CHANGED
|
@@ -33,18 +33,4 @@ Final checkpoint: RWKV-4-Pile-1B5-20220903-8040.pth : Trained on the Pile for 33
|
|
| 33 |
* SC2016 acc 68.73%
|
| 34 |
* Hellaswag acc_norm 52.48%
|
| 35 |
|
| 36 |
-
Preview checkpoint: RWKV-4-Pile-1B5-20220822-5809.pth : Trained on the Pile for 240B tokens.
|
| 37 |
-
* Pile loss 2.0518
|
| 38 |
-
* LAMBADA ppl 7.14, acc 56.36%
|
| 39 |
-
* PIQA acc 71.71%
|
| 40 |
-
* SC2016 acc 68.15%
|
| 41 |
-
* Hellaswag acc_norm 52.04%
|
| 42 |
-
|
| 43 |
-
Preview checkpoint: RWKV-4-Pile-1B5-20220814-4526.pth : Trained on the Pile for 187B tokens.
|
| 44 |
-
* Pile loss 2.0635
|
| 45 |
-
* LAMBADA ppl 7.34, acc 55.64%
|
| 46 |
-
* PIQA acc 71.44%
|
| 47 |
-
* SC2016 acc 68.25%
|
| 48 |
-
* Hellaswag acc_norm 51.60%
|
| 49 |
-
|
| 50 |
## Note: 4 / 4a / 4b models ARE NOT compatible. Use RWKV-4 unless you know what you are doing.
|
| 33 |
* SC2016 acc 68.73%
|
| 34 |
* Hellaswag acc_norm 52.48%
|
| 35 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
## Note: 4 / 4a / 4b models ARE NOT compatible. Use RWKV-4 unless you know what you are doing.
|